To make Firefox start up faster, you must clean your temporarily internet files: in Internet Explorer > Tools > Internet Options, on the "General" tab there is a section called "Browsing history". Click the "Delete" button, under "Temporarily internet files" click "Delete files" and then start Firefox. It should now open up faster.
It is also advisable to delete your Windows temporarily files.
This appears to be a known bug and should be fixed in Firefox 3.5.1.
To delete all such temporarily files and other useless junk, you may want to use History Killer Pro for which I still have a few free license keys to give to whomever comments.