What do you think about adding this to the feed, now that the audio stream is gone? [update: I have written a script that downloads, resamples, and uploads to Othernet]. It is quite easy to hear at low bitrate and based on transfer calculators would only take about 15-16 minutes to transfer via the feed. Feed replaces file daily or multiple times a day (same filename reused, ex: voa_news.mp3).
Its 293KB for a five minute global news report. Thoughts?
.mp3 files are now allowed.
Sure, this makes sense. Is there some automated process that’s can post the mp3 to GitHub?
I’m sure one can be made. What is the source routine for other pages you scrape? Is it all feeding to a GitHub source? A timed scrape with a LAME mp3 re-encode on a cron job could probably do the trick.
The news files are created from RSS feeds, but those are created in an intermediary server, which then pushes the files to the broadcast system. Using Github as a source of files set to ingest periodically is already operational, though. The feature was created for user to upload files, but it hasn’t been rolled out yet. We can easily use the same system for audio clips.
Can you tell us again, how you used to feed Othernet Radio from the VOA on-line feed. Ken
It was just the standard live stream published by VOA, but transcoded with Opus to get it down to about 8kbps. That audio stream came in over the broadcast and was then transcoded again into an mp3 stream, which is all browsers natively supplier.
The software that turned the stream back into mp3 was Shine:
I worked on this a bit last night. I have the feed ingestion working, as well as the conversion process to a highly optimized MP3 format (low-pass filtering, high compression, optimal bitrate,etc). I am waiting on some info from Syed to complete the process (stream injection routine).
Update #1
Heard back from Syed. I have a time routine that is pulling, converting, and uploading to the feed. Just going to work with Syed to see where it is hosted/run from.
Here is a test result of our latest efforts.
(Only 246.0 KB)
So after much experimentation we have managed to create a setup to digest audio content into a format that is both bandwidth conscious and an improvement in sound quality over previous methods and supported by DC3 (Chrome, Firefox and Edge). At current broadcast speeds, it should take only 12min 58sec to traverse data stream ( %0.9 of a 24hr stream cycle).
Plans are to add this (and possibly other) audio feeds back to the Othernet data stream soon. Note: these will be delivered as files you can open with the File Manager app not via the Radio app as done previously.
a similar 5 minute bulletin from NPR is 44k compressed with codec2/1200 (ok speech quality) and 87k (fine speech quality) with codec2/2400. codec2 can probably be even better quality in the future using machine learning (LPCnet). Problem is that the current skylark lacks the software to decompress c2 audio. At the moment it converts opus to mp3 for browser consumption. Im currently working on that problem.
Original Source File NPR News (Stereo)
Compressed to codec2/2400 (Mono) and rendered again as mp3
Compressed to codec2/1200 (Mono) and rendered again as mp3
And here are the original files for c2enc/c2dec for your own tests. GZipped so the forum will allow me to attach them. You cannot compress these much further.
nprnews_2400.c2.gz (76.6 KB)
nprnews_1200.c2.gz (42.7 KB)
Takes a minute for my brain to tune in, but the /1200 is perfectly usable for this kind of content for me. Quite amazing actually!
The goal (at this stage) is to deliver a file format that can be played natively on the DC3 and consequently Edge, Firefox, and Chrome. The examples above, as you stated, were converted back to MP3 for playback and are 2 MB to 3 MB at that size. OPUS (used in my previous post, at 246KB, is in its final form prior to .gz compression) is natively supported by DC3 and browsers listed above. Note: the extension of the file is renamed to MP3 to get the music player in DC3 and SkyLark to launch it. (a bit of a trick, but it does work).
If codec2 can use be used in such a way that you can just open it natively in the browser, fantastic! If not, then the extra steps required to listen to the file may be an issue for most users. It sounds great as an MP3 (2-3MB), but I haven’t listened to it as a .c2(44-87KB) file as I don’t have a player capable of decoding it at the moment.
@caveman99, is there any way to get a browser to play .c2 files? Or a way to get the DC3 ,at current firmware level, to convert .c2 files to mp3 files for playback? This would be best case if possible. If not, the c2dec converter should definitely be included in the Raspberry Pi project being considered as UI front end for DC4.
Being able to data stream the smaller .c2 and then having the PI convert it to MP3 for consumption would be a great generational improvement for Othernet on DC4+Pi.
@caveman99 Thanks for working on this too!!
I presume these approaches work on individual audio packages, and have to manually applied to generate the content you are sharing.
Before Othernet simply put up an MP3 audio stream and just let it play live. I believe the key here is to be able to get automatic selection of audio packets (like automatically get the hourly VOA newscast), then process it and upload it into the broadcast carousel. Is that possible? Ken
@kenbarbi yes. Working with @Syed on this. The files would arrive like any other content being delivered on the data-stream. In this case a file called voa_news.mp3. The stuff performed on the injest server would be download of the source .mp3 daily, conversion to reduced bandwidth format, then injection into the data-stream. Current thinking is the data stream won’t be split into audio/data as it was before. It will just all be treated as data file downloads. Which is better in my opinion, as you don’t have to tune in at a particular time to listen. The files are recorded for you automatically to consume when you wish. The end user won’t have to anything special, simply open the file in DC3 (Skylark).
@caveman99 NPR content is generally not license-free/available for redistribution. But I’m happy to add it if I’m wrong about that.
The utility value and marketing value of providing a simple satellite radio service should not be underestimated, especially free satellite radio. @Cheaha suggested that the TTS application Ivona could be used to create a quasi-audio service from just textual content.
Although the company Ivona was recently acquired by Amazon, the Android pkg is available and functions offline.
It’s a whole lot easier to source license-free text content than audio content.
@syed i was just using NPR as an example, as the source was 44khz/128kbit instead of the 32kbit for the VOA Newscast, which initially gave horrible results in C2. I have no intention to put NPR on the broadcast carousel.
My current approach is twofold: trying to find a smaller file format for transmission in the data stream, which then will be re-expanded on the DC3 after reception, pretty much lik bz2 content. At the moment there is a bunch of processes listening for final reception of compressed files and decompressing/moving the content in place.
In the long run these narrower streams could mean a return of live-radio, which is currently transcoded from OPUS back to MP3 in the DC3 audio server and provided on Port 8001 as a stream. Replacing that with C2 Audio would only mean a replacement of the audio server binary, or really substituting it with a netcat|s2dec|mp3enc|netcat pipe.
@caveman99 Another thing to keep in mind is the limited resources available on the ESP32-S2. Over time, I expect the number of receivers with limited CPU/RAM to dwarf the ones running Linux. There is still some CPU/RAM left on the DC4, but not tons.
Ideally codec2 could come in over the air and then be transcoded to mp3 on the Dreamcatcher 3 and 4. That’s the ideal.