Memory / optimization issues

I am working on a complex script that can handle up to 500,000 records. Here's my question.

Basically my code will parse a text file to get each of those 500,000 or so records. Each entry will have a category, my code will have to check if a new table entry was categories

created for that category during that particular processing, and if not, it will create that entry.

So I have 2 options:

1) I am storing an array of keys => values ​​containing the category name and ID, so I could do this:

if (array_key_exists($category,$allCategories))
   $id=$allCategories[$category];
else
{
   mysql_query("INSERT INTO categories (procId,category) 
                       VALUES ('$procId''$category')");
   $id=mysql_insert_id();
   $allCategories[$category]=$id;
}

      

        
        
        
      

    

2) Every time this text file is processed it gets its own process ID. So instead of checking for a variable $allCategories

that might grow to have 100,000 entries, I could do this:

SELECT id FROM categories WHERE procId='$procId' AND category='$category'

      

        
        
        
      

    

The downside here is that this query will run for each of the 500,000+ records. Whereas the disadvantage of storing all categories in an array is that I might lose memory or the server might crash.

Any thoughts?