Shop OBEX P1 Docs P2 Docs Learn Events
all of Wikipedia on the propeller? — Parallax Forums

all of Wikipedia on the propeller?

Zap-oZap-o Posts: 452
edited 2015-04-14 13:18 in General Discussion
I am designing a robot that hopefully can be "off line" much of the time and with that said I would like for it to remain useful.

I am dreaming of installing Wikipedia onto a SD card in hopes my robot could look things up for me. For a PC this open source program works fantastic - I am using it now on my laptop. Its called XOWA http://xowa.sourceforge.net/ an open source wiki browser.

Is there any negative thoughts about porting it to spin? Would the propeller be to slow indexing 4 gigs of wiki material? Has anyone ever tried this?

Comments

  • Heater.Heater. Posts: 21,230
    edited 2015-04-14 10:35
    Don't even think about it.

    The work required to port that to Spin would be huge.

    Not enough memory.

    Slow as hell.

    It's impossible.

    A Raspberry Pi or other such system might handle it. And be cheaper.
  • LoopyBytelooseLoopyByteloose Posts: 12,537
    edited 2015-04-14 12:01
    Ummm.... how much spece exactly would be required to download all of Wikipedia, if one was allowed to so do?

    It looks like you would require at least a few hundred gigabites.

    And where would you store it???
    How would you manage updating it???

    It seems you would need a hard disk. Try a Cubieboard at www.cubieboard.org and forget the Raspberry Pi (most of them don't have a SATA interface). It is likely to be a neater solution (less wires to and from things).
  • mindrobotsmindrobots Posts: 6,506
    edited 2015-04-14 12:11
    From the XOWA Sourceforge:

    Hardware

    CPU: 1.5 GHz minimum; 2.5 GHz recommended
    Memory: 512 MB minimum; 2 GB recommended
    Storage: varies per wiki.
    Space required during initial import: multiply the dump file by 8. For example, for English Wikipedia, the dump size is 10 GB. You should at least have 80 GB space free for the import process
    Space required when completed: multiply the dump file by 2.5. For example, for English Wikipedia, the dump size is 10 GB. When done, your English Wikipedia will be 25 GB.

    Buy your robot a Chromebook and let it browse online! :D
  • Zap-oZap-o Posts: 452
    edited 2015-04-14 12:12
    Downloading all of Wikipedia without high quality phots is about 4 gigs, mine is stored on an SD card. I have done this on the PC and I am really impressed. I still dont see why the propeller would not be able to do this, speed is the only issue imo. Why not try?

    If I could do this on a propeller server (spinneret) this would be very cool and useful. I also hate it when people say dont even think about it! :)

    Here is a reader written for devices running Fourth. http://en.wikipedia.org/wiki/WikiReader <-- if this microprocessor can do it surly a propeller can!
  • localrogerlocalroger Posts: 3,451
    edited 2015-04-14 13:18
    Actually a wikireader does exactly what Zap-o proposes, with a fairly limited CPU and for 100 hours on a set of alkaline batteries. You can even hack a connector for the serial port and use a PropPlug to program it in Forth.

    http://en.wikipedia.org/wiki/WikiReader

    So it's possible. Would probably require C with some external RAM to get enough program functionality to both read the SD card file system and do all the indexing. Probably be slow and a lot more battery hungry than the wikireader, but I wouldn't say impossible.

    On edit -- Jinx! Note that while the wikireader can be programmed in Forth, as this was used as a debugging / scripting language by the developers, the actual Wikireader software is written in C and the developers admit that Forth would probably be too slow to be practical for the main functionality.
Sign In or Register to comment.