testing spider

mctrivia

many months ago I said I was going to try to archive the parallax forum january 1st.

well it did not happen because other things came up. I have goten back on this project and starting now I am attempting my first test run of my spider. Will get back to you when it finishes.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
24 bit LCD Breakout Board now in. $24.99 has backlight driver and touch sensitive decoder.

If you have not already. Add yourself to the prophead map

mctrivia

anyone remember how to kill a thread on a linux box via ssh? i forgot to write an abort script into the spider and do not need it to download the entire forum for a test.

ps command tells me it is pid 19789 that i will want to kill

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
24 bit LCD Breakout Board now in. $24.99 has backlight driver and touch sensitive decoder.

If you have not already. Add yourself to the prophead map

mctrivia

well test results are in. and it fails in 2 places.

1) downloading of avatars fails. my test code works so the problem must be in the spiders parsing of the url.

2) parsing of post body fails.




I have attached the template i am using. anyone want to make a better one? I can define any more dynamic html tags you need. final site layout will have the following directory structure:

main
*images
  *avatar
  *bb
*1
*3
*5
...
*36
*40
*includes

1 to 40 is the number of each forum.



P.S. I remembered how to kill the thread.
kill -9 19789

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
24 bit LCD Breakout Board now in. $24.99 has backlight driver and touch sensitive decoder.

If you have not already. Add yourself to the prophead map