For some reason, downloading an entire folder from within a symlink'd directory is not working. This used to work... so I'm not entirely sure what broke. This is something I'll have to actually reproduce locally and debug to figure out. But, the rest of it I think is still functional.
Looks like Github is easier to use. Maybe when you work out the bugs it will be useful.
I certainly understand bugs making this.. less than useful . Is that all you're referring to? Are there usability issues with the working parts of it that you'd like to see addressed? Even if it's already been listed in the GitHub issue tracker or here on the forums, please feel free to speak up. This is definitely a case of "squeaky wheel gets the grease."
Propeller users are interested in your approach, David. I think that you're facing many of the issues we did with OBEX: design, coding, maintenance, bug hunts. It becomes a lot of work, quickly.
We're not sure how to tie in your efforts. I think you need to get together with Jeff Martin at some stage and discuss if there's a fit/use/link approach we should use.
I'm not a fan of Github at all, but I can go to github Obex, All, or a subcategory and open a spin file. sometimes the raw option doesn't work and you have to save it with the line numbers, then delete them later, some objects are missing. On yours I open some white page with a tree on my left, click on libraries, or docs, etc. and it shows a two line readme. If I use my old Win7 it just shows a plain white page and nothing else. I guess I could save every OBEX object and then post them on the forum. There should be a new forum category and call it OBEX. And have each spin object as a post. Like Basic I2C so all you do is click on it and it opens your spintool. (Good Idea Bob)
I'm not a fan of Github at all, but I can go to github Obex, All, or a subcategory and open a spin file. sometimes the raw option doesn't work and you have to save it with the line numbers, then delete them later, some objects are missing. On yours I open some white page with a tree on my left, click on libraries, or docs, etc. and it shows a two line readme. If I use my old Win7 it just shows a plain white page and nothing else. I guess I could save every OBEX object and then post them on the forum. There should be a new forum category and call it OBEX. And have each spin object as a post. Like Basic I2C so all you do is click on it and it opens your spintool. (Good Idea Bob)
What do you mean the raw option sometimes doesn't work? Does it consistently not work for certain filetypes or something, or does it appear to work or fail randomly? What happens when it doesn't work - does it just show you a blank page, or what? What web browser are you using, and have you tried a different one? Does it work if you right-click the "Raw" button and then pick "Save Link As..."?
Depending on the object post the Raw option works 50/50. Sometimes it shows the code perfectly other times its just a page of random characters. I still use my Win7, but I have Win10, and my RasberryPi. I have my Mac but rarely use it, just bought it to do Xcode.
@DavidZemon
I stumbled on this answer about accessing single files from github:
This is an old question nowadays you can get to the source through: https://raw.githubusercontent.com
https://raw.githubusercontent.com/ user / repository / branch / directory / file
e.g. https://raw.githubusercontent.com/sachatrauwaen/OpenContent-Templates/master/Bootstrap3Accordion/schema.json
I assume one could use this to get any files from the repo, one at a time fairly easily. If I knew enough about webdev, I would attempt to write something to use this, but I don't...
I figured either you knew this already, or would like to know about it.
@DavidZemon
I stumbled on this answer about accessing single files from github:
This is an old question nowadays you can get to the source through: https://raw.githubusercontent.com
https://raw.githubusercontent.com/ user / repository / branch / directory / file
e.g. https://raw.githubusercontent.com/sachatrauwaen/OpenContent-Templates/master/Bootstrap3Accordion/schema.json
I assume one could use this to get any files from the repo, one at a time fairly easily. If I knew enough about webdev, I would attempt to write something to use this, but I don't...
I figured either you knew this already, or would like to know about it.
I am aware of it. I actually thought I was using it already. Apparently not though. For displaying a file "preview", obex.zemon.name makes a call out to https://obex.zemon.name/api/downloads/<path> to retrieve the file content. This same service can serve both files and folders. If the service detects that the requested "<path>" is a folder or symlink, it will zip it up and respond with that archive. If it is a regular file, it will respond with the file's contents.
A good thing about a spin post is, if you click on it the spintool opens up. Anything on Github you have copy, paste, or save it. If you can get your Obex to open the spintool automatically, that would be much better than Github.
Depending on the object post the Raw option works 50/50. Sometimes it shows the code perfectly other times its just a page of random characters. I still use my Win7, but I have Win10, and my RasberryPi. I have my Mac but rarely use it, just bought it to do Xcode.
If you save that page of seemingly random characters anyway and open it in Propeller Tool, does it load correctly? It's probably just encoded as UTF-16, which your browser probably doesn't know how to display but should be able to save correctly.
So, this is getting a bit of a revival. Who knows how long it will last, but at the moment, I'm very intrigued by Rust.
In light of that, I've been looking for something to do with Rust. Converting my existing Python version of the Obex server to Rust would be a great way learn the language while also adding a few enhancements to the app that should make it a bit more resilient than previously.
So far, I've re-written the tree-traversal aspect, which allows you to view the tree structure (but not view or download any contents) and I've added a background thread that polls github every 30 minutes for updates. If the app starts and the repo doesn't exist on the filesystem, it will be cloned. These last two bits are the "few enhancements" that I mentioned which "should make it a bit more resilient." This is because my initial version of the obex would crash every time the Linux server was restarted, because I had manually cloned the repository into /tmp and every reboot would require me to log in and re-clone the project. Obviously, I wasn't doing that, and it wasn't long before it became useless vaporware.
I was also able to serve the static content from the Rust server much more easily than from the Python server. This isn't to say the Rocket library in Rust is amazingly more powerful and better than Django... but it happened to make this one thing much easier.
With the server now written in Rust, the Docker image is dramatically smaller. With the Rust server, static content, and git executable, the uncompressed tarball is 35M and compressed it's only 16M:
david@balrog:~$ docker save -o obex.tar davidzemon/obex
david@balrog:~$ l -lh obex.tar
-rw------- 1 david david 35M Jun 2 11:15 obex.tar
david@balrog:~$ gzip obex.tar
david@balrog:~$ l -lh obex.tar.gz
-rw------- 1 david david 16M Jun 2 11:15 obex.tar.gz
And loading the web page is almost instantaneous now. In Python, it required approximately half a second to traverse the obex and send the 1.7MB tree in the HTTP response. That is down to 70-80ms with Rust.
Source for the new Rust version of the server is here: https://github.com/DavidZemon/obex-server-rust
I won't bother deploying it to obex.zemon.name until I've implemented the download functionality, as it is nothing more than a novelty until then. But once I do, I'll be sure to drop another update.
Rust version is now at feature parity with the Python version, plus it has the above-mentioned enhancements. All that's left is to get the CI server up and running again and deploy the new version
Server is back up and running.
It should stick around this time.
As far as I can tell, everything is working perfectly, and should continue to do so long into the future.
@"Jeff Martin",
This new Rust server is more "complex" than the Python version, but you might have an employee or two that would like to learn Rust, so maybe that's actually a good thing?
Now that the site is up and running reliably, you can at least evaluate it and decide whether or not it's something you guys are interested in. If you are interested in it, but Rust is a turn off, then I can backport the new features to old Python version again. I'm thrilled with how the Rust version turned out, but I don't think I'd actually recommend someone build a web server with Rust unless performance is really important .
For anyone who would like to try running this locally:
docker run -it --rm -p 8000:8000 --name obex davidzemon/obex
Just a final update here: I'm turning off this server and removing the link from my sig, now that an official Obex from Parallax is up and running again.
Comments
Ah! Problem solved. You just need to use a real computer
dgately
Ken Gracey
https://obex.zemon.name is now working again (until the next reboot).
For some reason, downloading an entire folder from within a symlink'd directory is not working. This used to work... so I'm not entirely sure what broke. This is something I'll have to actually reproduce locally and debug to figure out. But, the rest of it I think is still functional.
I certainly understand bugs making this.. less than useful
We're not sure how to tie in your efforts. I think you need to get together with Jeff Martin at some stage and discuss if there's a fit/use/link approach we should use.
Ken Gracey
What do you mean the raw option sometimes doesn't work? Does it consistently not work for certain filetypes or something, or does it appear to work or fail randomly? What happens when it doesn't work - does it just show you a blank page, or what? What web browser are you using, and have you tried a different one? Does it work if you right-click the "Raw" button and then pick "Save Link As..."?
I stumbled on this answer about accessing single files from github:
The e.g. in that quote loads up the raw file in the browser. This one works too: https://raw.githubusercontent.com/parallaxinc/propeller/master/libraries/community/p2/All/jm_1-wire/jm_1-wire.spin2
I assume one could use this to get any files from the repo, one at a time fairly easily. If I knew enough about webdev, I would attempt to write something to use this, but I don't...
I figured either you knew this already, or would like to know about it.
I am aware of it. I actually thought I was using it already. Apparently not though. For displaying a file "preview", obex.zemon.name makes a call out to https://obex.zemon.name/api/downloads/<path> to retrieve the file content. This same service can serve both files and folders. If the service detects that the requested "<path>" is a folder or symlink, it will zip it up and respond with that archive. If it is a regular file, it will respond with the file's contents.
If you save that page of seemingly random characters anyway and open it in Propeller Tool, does it load correctly? It's probably just encoded as UTF-16, which your browser probably doesn't know how to display but should be able to save correctly.
So, this is getting a bit of a revival. Who knows how long it will last, but at the moment, I'm very intrigued by Rust.
In light of that, I've been looking for something to do with Rust. Converting my existing Python version of the Obex server to Rust would be a great way learn the language while also adding a few enhancements to the app that should make it a bit more resilient than previously.
So far, I've re-written the tree-traversal aspect, which allows you to view the tree structure (but not view or download any contents) and I've added a background thread that polls github every 30 minutes for updates. If the app starts and the repo doesn't exist on the filesystem, it will be cloned. These last two bits are the "few enhancements" that I mentioned which "should make it a bit more resilient." This is because my initial version of the obex would crash every time the Linux server was restarted, because I had manually cloned the repository into /tmp and every reboot would require me to log in and re-clone the project. Obviously, I wasn't doing that, and it wasn't long before it became useless vaporware.
I was also able to serve the static content from the Rust server much more easily than from the Python server. This isn't to say the Rocket library in Rust is amazingly more powerful and better than Django... but it happened to make this one thing much easier.
With the server now written in Rust, the Docker image is dramatically smaller. With the Rust server, static content, and git executable, the uncompressed tarball is 35M and compressed it's only 16M:
And loading the web page is almost instantaneous now. In Python, it required approximately half a second to traverse the obex and send the 1.7MB tree in the HTTP response. That is down to 70-80ms with Rust.
Source for the new Rust version of the server is here: https://github.com/DavidZemon/obex-server-rust
I won't bother deploying it to obex.zemon.name until I've implemented the download functionality, as it is nothing more than a novelty until then. But once I do, I'll be sure to drop another update.
Rust version is now at feature parity with the Python version, plus it has the above-mentioned enhancements. All that's left is to get the CI server up and running again and deploy the new version
Server is back up and running.
It should stick around this time.
As far as I can tell, everything is working perfectly, and should continue to do so long into the future.
@"Jeff Martin",
.
This new Rust server is more "complex" than the Python version, but you might have an employee or two that would like to learn Rust, so maybe that's actually a good thing?
Now that the site is up and running reliably, you can at least evaluate it and decide whether or not it's something you guys are interested in. If you are interested in it, but Rust is a turn off, then I can backport the new features to old Python version again. I'm thrilled with how the Rust version turned out, but I don't think I'd actually recommend someone build a web server with Rust unless performance is really important
For anyone who would like to try running this locally:
Then visit http://localhost:8000 and you should see the same thing as https://obex.zemon.name.
Just a final update here: I'm turning off this server and removing the link from my sig, now that an official Obex from Parallax is up and running again.