php - This code needs to loop over 3.5 million rows, how can I make it more efficient? -


I have a CSV file that has 3.5 million codes.
I should tell that it is only going to happen this once.

Looks like CSV 

  age9tlg, rigfh34, ...

This is my code: 

  ini_set ('max_execution_time', 600); Ini_set ("memory_limit", "512m"); $ File_handle = fopen ("Weekly CSV", "R"); While {$ line_of_text = fgetcsv ($ file_handle); if (is_array ($ line_of_text) foreach ($ line_of_text $ as col) {if (($ col)) {mysql_query ( Enter in the `action_6_weekly` values ​​('$ col', '') or die (mysql_error ());}} and {if (empty ($ line_of_text)} {mysql_query (" `action_6_weekly` values ​​(' $ Line_of_text ',' ') or die (mysql_error ());}}} Fclose ($ file_handle);

Is this code going to die partly on me? My memory and maximum execution time will be quite high?

NB: This code will be run on my localhost, and the database will be the same C is at the same time, therefore latency is not a problem.

Update:
Here is another possible implementation, it performs in bulk insertion of a 2000 record $ I = 0; $ Vals = array (); while ((FIF ($ file_handle)) $ {$ ($ file_handle)) Line_of_text = fgetcsv ($ file_handle); {If_array ($ line_of_text) foreach ($ line_of_text $ as col) {if (! Below ($ col) {if ($ i> 2000) {$ vals [] = "('$ col', '')"; $ I ++; } And {$ vals = implode (',', $ vals); Mysql_query ("Enter value in action_6_wek $ vals") or die (mysql_error ()); $ Vals = array (); $ I = 0; }}} And {if (empty ($ line_of_text)} {if ($ i> 2000) {$ vals [] = "('$ line_of_text', '')"; $ I ++; } And {$ vals = implode (',', $ vals); Mysql_query ("Enter value in action_6_wek $ vals") or die (mysql_error ()); $ Vals = array (); $ I = 0; }}}} Fclose ($ file_handle);

If I want to use this method, can I determine the highest value to include it at one time?

Update 2
So, I found that 

  load data local INFILE 'C: \\ xampp \ \ Htdocs \\ Weekly 'csv' can be used by the ';' set by the 'action_6_quickly' field '' Avoid '\\' fixed by lines, '(`code`)

But this issue is now, I was wrong about the CSV format, this is actually 4 Code and then a line break, hence freefllogy, qporlfg, vcalpfx, rplfigc, vapworf, flofigx, apqoeei, clxosrc,
...

so I should be able to specify two rows ending Given - this question has been equated.

Update 3
Setting it to insert bulk of 20k rows 

  while ($ feof ( $ File_handle)) {$ val [] = fgetcsv ($ file_handle); $ I ++; If ($ i == 20000) // // insert // set $ I = 0; // $ val = array ();}} // (for the last few rows which do not go up to 20k

but this dies at this point because for some reason $ There are 75 lines in the vine, and why consider?
Note that the above code is simplified.

I suspect it would be the popular answer, but if I run on a CSV file it will be my php applicati, of course it is more optimized than what you have to do in php


-

Comments

Post a Comment

Popular posts from this blog

Eclipse CDT variable colors in editor -

I hope this is a good question to ask here. This programming is related, so I thought it would be better than the superuser, so I am using the CDT C ++ Eclipse plugin. I know how the editor changes colors for some things But is there a way to do that all the variables are different colors? For example in KDEValue, it sets the local variables and square variables in different colors and bold square squares. Can I copy it to CDT? Eclips provides some level of color customization for text editor syntax You can see it on the following: There are many elements, including the Window> Preferences> C / C ++> Editor> Syntax color code that can be changed, assembly, Comments, Preprocessor and Doxian Under the code, it is possible to change the color of some variable types, such as global variables, local variable announcements, local variable references, and parameter variables. But I do not think the code is capable of changing the color of each variable declared....
Read more

AJAX doesn't send POST query -

$ (दस्तावेज़) .ready (function () {function ajaxselectrss (rssurlvar) {var ajaxRequest; // वेरिएबल जो अजाक्स को संभव बनाता है! कोशिश करें कि {// ओपेरा 8.0+, फ़ायरफ़ॉक्स, सफारी अजाक्स अनुरोध} = नया एक्सएमएलएचटीपीआरइवेस्ट ();} कैच (ई) {// इंटरनेट एक्सप्लोरर ब्राउज़र्स {ajaxRequest = new ActiveXObject ("Msxml2.XMLHTTP");} पकड़ो (ई) {try {ajaxRequest = new ActiveXObject ("Microsoft.XMLHTTP");} कैच (ई) {// कुछ गलत चेतावनी ("आपका ब्राउज़र तोड़ दिया!"); वापसी झूठी;}}} // एक बनाएं फ़ंक्शन जो सर्वर से भेजे गए डेटा प्राप्त करेंगे ajaxRequest.onreadystatechange = function () {if (ajaxRequest.readyState == 4) {var ajaxDisplay = document.getElementById ('news'); ajaxDisplay.innerHTML = ajaxRequest.responseText;}} / / Var rssurlvar = $ (this) .attr ("title"); var queryString = "rurl =" + rssurlvar; var urltofile = "rssget.php"; ajaxRequest.open ("पोस्ट", urltofile, सच); ajaxRequ Est.setRequestHeader ("सामग्री-प्र...
Read more

wpf - Custom Message Box Advice -

Well, I'm using a custom window box that comes with some custom control boxes that accompany the text The producers who are displayed are called. I have a defined event, which has been subscribed through the original class, it gets burnt when clicking on the button. However, I can not see how to use it effectively, I would like to return one yes, whether yes or not, but obviously my code will continue to execute, so this Below is some example code to clarify the problem that the method has subscibed by clicking on the button. Message box window Public partial class custom message box: window {public representative message messagebox object sender, event events e); Public event messagebox handler messagebox document; Public Customs Box () {Initial group (); } Public Customsbox (String Message) {InitializeComponent (); This.txtdescription.Text = Message; } Public Customsbox (String Message, String Title, String First BTNtext) {Initialization (); This.lbltitle.Content =...
Read more