Andrew Grathwohl 2020-08-13T19:32:53-04:00 https://grathwohl.me Working with RTSP in FFmpeg 2020-06-23T08:04:00-04:00 https://grathwohl.me/2020/06/23/RTSP-FFmpeg <p>Recently, I’ve found myself situated among a <a href="https://news.yahoo.com/whats-sound-not-just-york-025006561.html?soc_src=hl-viewer&amp;soc_trk=tw">great fireworks battle</a> in Brooklyn, NY. As we continue to work out the details about the heightened fireworks activity across the United States, I felt it was important that I document and share my experience to contribute what I can to the ongoing conversation. So, I grabbed an old IP camera I had in a corner gathering dust, placed it outside on my fire escape, and fired it up to get a decent shot of the night sky in my neighborhood.</p> <h2 id="ip-camera-trickiness">IP Camera Trickiness</h2> <p><img src="/rtsp-control.jpg" alt="" /></p> <p>I did a little snooping around the control pannel software on the camera to discover that the RTSP protocol is used to transport the signal over ethernet. Being an FFmpeg stan, I was confident that I would be able to quickly grab and relay my <a href="https://store.ui.com/collections/unifi-protect-cameras/products/unifi-video-camera-g3-dome">UVC-G3-DOME</a> signal right away. However, I failed to consider the contexts posed by the delivery transport. So, I was a bit surprised to discover that typing a typical <code class="language-plaintext highlighter-rouge">ffmpeg</code> test command to ingest this camera’s feed resulted in some really weird error logs and a very corrupt video output.</p> <div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffmpeg <span class="nt">-re</span> <span class="nt">-i</span> rtsp://192.168.3.55:554/s0 <span class="nt">-c</span>:v libx264 <span class="nt">-c</span>:a libmp3lame test.mkv </code></pre></div></div> <p><img src="/rtsp-corrupt.jpg" alt="" /></p> <p>The above command would be adequate for testing the feed from software like <code class="language-plaintext highlighter-rouge">obs</code> and from services like AWS Kinesis or Wowza Cloud. But this RTSP signal looked all sorts of weird when saved to disk.</p> <h2 id="how-to-rtsp-in-ffmpeg">How To RTSP in FFmpeg</h2> <p>Upon doing some research, I discovered that <code class="language-plaintext highlighter-rouge">ffmpeg</code> has rich support for the <a href="http://ffmpeg.org/ffmpeg-all.html#rtsp">RTSP protocol</a>, but as a result there are some specific FFmpeg flags that must be utilized that I’d failed to include in my previous test command. <code class="language-plaintext highlighter-rouge">ffmpeg</code> requires the <code class="language-plaintext highlighter-rouge">-rtsp_transport</code> input flag to declare the transport used by the camera - TCP in my camera’s case. This new test command produced a most agreeable output.</p> <div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffmpeg <span class="nt">-re</span> <span class="nt">-rtsp_transport</span> tcp <span class="nt">-i</span> rtsp://192.168.3.55:554/s0 <span class="nt">-c</span>:v libx264 <span class="nt">-c</span>:a libmp3lame test.mkv </code></pre></div></div> <p><img src="/rtsp-clean.jpg" alt="" /></p> <h3 id="rtmp-relaying">RTMP Relaying</h3> <p>My ultimate use case is a common one - I wanted to be able to record the live feed to local disk as well as relay the stream to a YouTube live stream RTMP endpoint. To accomplish this, <code class="language-plaintext highlighter-rouge">ffmpeg</code> requires some specific output options declared to ensure compliance, as the direct RTSP signal from the camera is not in a compliant format.</p> <p>To resolve the RTMP compliance issue, I declared <code class="language-plaintext highlighter-rouge">-f flv -ar 44100 -r 30/1 -pix_fmt yuv420p</code> as output options for the RTMP portion of my <code class="language-plaintext highlighter-rouge">ffmpeg</code> command. Declaring the format as FLV ensures the output bytestream is ordered in a way that is compliant with YouTube’s RTMP endpoint. Setting an explicit 44.1kHz sampling frequency for our audio guarantees full-spectrum audio while lowering overall output bitrate. And finally, enforcing the YUV420P pixel format ensures that your camera’s colors are correctly transformed to match YouTube’s color space.</p> <p>The full command now looks like this:</p> <div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffmpeg <span class="nt">-re</span> <span class="nt">-rtsp_transport</span> tcp <span class="nt">-i</span> rtsp://192.168.3.55:554/s0 <span class="nt">-c</span>:v copy <span class="se">\</span> <span class="nt">-c</span>:a copy stream_capture.mkv <span class="nt">-c</span>:v libx264 <span class="nt">-b</span>:v 3M <span class="nt">-minrate</span>:v 3M <span class="nt">-maxrate</span>:v 3M <span class="se">\</span> <span class="nt">-bufsize</span>:v 6M <span class="nt">-g</span> 120 <span class="nt">-r</span> 30/1 <span class="nt">-x264-params</span> <span class="nv">keyint</span><span class="o">=</span>120:min-keyint<span class="o">=</span>60 <span class="se">\</span> <span class="nt">-c</span>:a libmp3lame <span class="nt">-ar</span> 44100 <span class="nt">-pix_fmt</span> yuv420p rtmp://YOUTUBE_RTMP_URL </code></pre></div></div> <p><img src="/fireworks.gif" alt="" /></p> <p>That’s definitely a firework exploding a few feet away from my apartment building! Nice.</p> The Best Free and Open Source Media Software 2020-05-27T11:51:00-04:00 https://grathwohl.me/2020/05/27/FOSS-Media-Software <p>The FOSS ecosystem’s many benefits present only a few minor annoyances for sysadmins - the most significant of which is the need for research, as well as trial-and-error, when investigating new software to employ in your stack.</p> <p>If you are in need of suggestions for FOSS media software, this article is for you.</p> <h2 id="tldr">TLDR</h2> <div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>pacman <span class="nt">-S</span> sox mpv vapoursynth ffmpeg mkvtoolnix-cli graphicsmagick </code></pre></div></div> <h3 id="audio">Audio</h3> <h4 id="sox-lgpl">SoX (LGPL)</h4> <p><a href="http://sox.sourceforge.net/">SoX</a> - or, Sound eXchange - is an LGPL application for processing sound. It supports every free (and some non-free) audio codecs, and employs by far the best resampler out there today, <code class="language-plaintext highlighter-rouge">libsoxr</code>. Below, we see SoX totally annhilating expensive, proprietary “professional” audio software, Ableton Live, in a sampling rate conversion test.</p> <p><img src="/sox-src.jpg" alt="" /></p> <h3 id="containers">Containers</h3> <h4 id="matroska">Matroska</h4> <p>If you want to stay sane working with media in the FOSS realm, you’ll need to be using <a href="https://www.matroska.org/">Matroska</a>. A fully open and highly extensible container format, it is just about as free as free software gets.</p> <p>At a minimum, make sure you have <a href="https://mkvtoolnix.download/"><code class="language-plaintext highlighter-rouge">mkvtoolnix</code></a> laying around for easy and flexible muxing, demuxing, and metadata tasks.</p> <h3 id="images">Images</h3> <p>For many years, the undisputed FOSS image processing king was <code class="language-plaintext highlighter-rouge">ImageMagick</code>, however over the years the project has failed to address meaningful inefficiencies and bugs within the software. This spawned the <code class="language-plaintext highlighter-rouge">GraphicsMagick</code> fork, which I would recommend to most users. Only those who need its deep colors features should consider sticking with the old and tired ImageMagick.</p> <h4 id="graphicsmagick-mit">GraphicsMagick (MIT)</h4> <p>If you are needing to process images in batch, <a href="http://www.graphicsmagick.org/">GraphicsMagick</a> is your starting shortstop. Its use of OpenMP means that CPU-bound tasks scale linearly as processor cores are added.</p> <h3 id="playback">Playback</h3> <h4 id="mpv">mpv</h4> <p><img src="/mpv.jpg" alt="" /></p> <p>The VideoLan folks have always done a fantastic job with the VLC player, however if you are serious about working with media you’ll want to move on over to the new king of FOSS media playback, <code class="language-plaintext highlighter-rouge">mpv</code>. Those 10-bit color anime folks love it, so you know it must be good.</p> <p><code class="language-plaintext highlighter-rouge">mpv</code> has a world-class plugin system that enables users to contribute scripts and plugins in either Lua and NodeJS. This is very useful for media analysis and for media processing automation.</p> <h3 id="video">Video</h3> <p>Video encoding and DSP today is largely centralized within the FFmpeg project, however more advanced operations will require additional tooling.</p> <h4 id="ffmpeg-lgplgplnon-free">FFmpeg (LGPL/GPL/non-free)</h4> <p><a href="https://ffmpeg.org">FFmpeg</a> is one the most significant and prolific FOSS projects out there today. Known as the “Swiss Army Knife” of multimedia coding, FFmpeg is an indispensible tool for your media workflow, and will likely be a dependency for building many other great media tools.</p> <p>You’d be pretty crazy to attempt any video work within the FOSS ecosystem without a recent build of FFmpeg.</p> <h4 id="vapoursynth-lgpl">VapourSynth (LGPL)</h4> <p><img src="/vs.jpg" alt="" /></p> <p>Many think that <em>if you can’t do it in FFmpeg, it probably can’t be done</em>, but that’s probably because they don’t know about <a href="http://www.vapoursynth.com/">Vapoursynth</a>.</p> <p>VapourSynth is a powerful open source Python framework for advanced video frame processing and other visual DSP operations. It’s got some of the best post-production and color tooling out there, and a really healthy community of video freak contributors who dogfood their own work in meaningful ways. It is in many ways a replacement for the aging, less-performant AviSynth framework.</p> <p>A few notes about VapourSynth… first, this is definitely software developed by <em>media engineers</em>, so you will discover bugs. Expect releases to come out very frequently, and definitely don’t skip reading the docs. Also, some operations on VapourSynth will take a <em>hilariously</em> long amount of time to complete, though the results are often worth the wait.</p> Distributed Media's Ancient Origins 2019-11-04T07:00:00-05:00 https://grathwohl.me/2019/11/04/Distributed-Media <blockquote> <p>This article originally appeared on the <a href="https://medium.com/ara-blocks">ARA Medium channel</a>.</p> </blockquote> <h2 id="how-a-two-million-year-old-event-helped-us-build-the-future">How A Two-Million Year Old Event Helped Us Build the Future</h2> <p>As 2019’s ongoing streaming media wars continue to heat up, technologists peering around the corner are beginning to see the distinct advantages in decentralizing and distributing our media workflows. The rise of AT&amp;T’s WarnerMedia and Oath’s transformation into Verizon Media signal that the world of content creation is already undergoing a significant shift in distribution practices.</p> <p>As a result, we are now entering a time where content is no longer <em>merely</em> <a href="https://www.youtube.com/watch?v=kums8SGXHoo"><em>king</em></a>. Content has become the <a href="https://www.emarketer.com/content/average-us-time-spent-with-mobile-in-2019-has-increased">soil under our feet</a>, the <a href="https://www.businessinsider.com/how-much-tv-do-americans-watch-2016-6">air we breathe</a>, and the <a href="https://everysecond.io/youtube">water we drink</a>. It ceases to serve as a mere authority in the world — it is now <em>the</em> authority of the world — and any person, group, or brand who wishes to reach its target audience must participate by means of content creation.</p> <p>Today, content creation is embraced by more people than ever before. That’s why at <a href="https://ara.one"><strong>Ara</strong></a>, we’re building out the distributed content network that will benefit all content creators, both new and seasoned, by removing third parties, reducing hosting costs, and providing a powerful new way to reward participation in content delivery.</p> <h2 id="the-inevitability-of-decentralization--distribution">The Inevitability of Decentralization &amp; Distribution</h2> <p>Early on, we saw that distributed digital content networks would be the wave of the future, and today, we see their coming decentralization as an inevitability.</p> <p>History has shown us that popular media tends to undergo two transitions as it evolves from a new art into a ubiquitous aspect to everyday life: first, it <strong>decentralizes</strong> from its initial authority; then, it becomes <strong>distributed</strong>. Once a media is <strong>distributed</strong>, the lines start to blur between producer and consumer. The evolution of video entertainment provides ample evidence in favor of this observation.</p> <p>After spending decades centralized around the movie theater chains, video entertainment began to <strong>decentralize</strong> in the late 1940s, thanks to the development of new technology that made video easier to capture, edit, and distribute. The television industry formed around these new innovations, and as a result, video’s customer base expanded significantly, dethroning the movie theater chains as the central authority in video. By 1952, there were 20 million television sets installed in homes throughout the United States, vastly exceeding the number of movie theaters.</p> <p>Of course, after cable TV had its go at expanding the industry even further, the internet continued video’s decentralization. New entrants like Netflix, YouTube and Twitch have brought both the cinema and broadcast TV alike into our homes, leading to today’s unquestionable arrival of <strong>distributed</strong> video. However, anyone who’s read the news lately would likely agree: our ambitions in distributed media far outpace society’s readiness for it.</p> <p>That is why the future will run on the infrastructure we build out today, otherwise we’ll struggle to meet our newfound content needs in a few short years. At <strong>Ara</strong> we’ve been hard at work designing a distributed framework specifically addressing content delivery. Throughout this process, we’ve found ourselves taking inspiration from some rather unlikely sources — among them, the discovery of fire. The reasons for fire’s timeless prevalence throughout human history serves as an ample springboard for figuring out how to encourage and reward participation in a future dominated by distributed media.</p> <h2 id="fire-the-ultimate-first-mover-advantage">Fire: The Ultimate First-Mover Advantage</h2> <p>Nearly two million years ago, some nameless genius made history by building the world’s very first campfire. That individual could have never dreamed of what they were making possible with this one act of brilliance. The discovery of fire did much more than help us cook our food, produce light amidst the darkness, and protect us from predators. Fire opened up the very first creative possibilities for all human beings, giving us a means to achieve brilliance in nearly every aspect of culture.</p> <p>Think about it: before we could huddle into small tribes, feeling sheltered and safe under the watchful eye of a roaring flame, how were we to ever come up with such relatively <em>frivolous</em> ideas as singing, storytelling, and dancing? Without the ability to control light, who would ever bother trying to draw, paint, or write?</p> <p>Mark Zuckerberg, Steve Jobs, and even Gutenberg all cower under the awesomely epic shadow cast by that one anonymous hero, who dragged all of our ancestors out of the caves and into the <a href="https://www.tesla.com/blog/introducing-software-version-10-0">Netflix-equipped self-driving electric cars</a> in which we find ourselves today.</p> <p>Even though fire has proved itself to be overwhelmingly valuable, no one central authority has ever controlled fire. In this sense, the control of fire was humanity’s very first form of distributed media, and the campfire is its earliest and most enduring implementation. By merely providing light and heat, campfires generate <em>community</em>, <em>participation</em>, <em>trust</em> and <em>safety</em> — precisely the same qualities that underpin successful distributed content networks.</p> <h2 id="community">Community</h2> <p>Gathering around the fire with friends and family is most certainly one of life’s basic pleasures. This is because for millennia, humans have relied on their communities to survive and thrive, and fire has always played an essential role within those communities. Reliance upon fire goes well beyond the obvious benefits of warmth, light, and protection — without it, early societies would have had a much harder time developing their cultural arts, traditions, and values.</p> <p>Community provides a fertile audience for the expression of ideas, the capability for individual specialization, and the pride gained through participating in something bigger than oneself. Successful distributed media protocols such as BitTorrent have always relied upon an active and welcoming community to maintain their dominance. Those communities came about by both projects’ decisions to open source their code, which enabled them to be available on nearly every platform imaginable, supported by every operating system, and implementable in all sorts of other useful projects. This in turn has led to innovations such as <a href="https://webtorrent.io/">WebTorrent</a>, breathing new life into this already outstanding web protocol.</p> <p>Ara has followed in these projects’ footsteps by <a href="https://medium.com/@arablocks/why-were-choosing-open-source-9df789230d0a">being an open source product</a> from day one. But there are many ways where we can, and should improve upon today’s state of affairs. Ara uniquely fosters community around digital content by rewarding users for helping to distribute the movies, shows, music, and games that they’ve purchased. Unlike the users of BitTorrent or Bitcoin, users of Ara know that every time they receive a reward for sharing with their peers, they’ve just helped their favorite content creators achieve success, too.</p> <h2 id="participation">Participation</h2> <p>While gathering around the fire is a great way to foster community, without modes of participation, individuals will begin to fall out from that community rather quickly. Consider how one must participate in keeping a campfire continually lit, by adding more kindling and stoking the flame. Since nobody can gather kindling and light a fire at the same time, specialization and division of labor quickly emerged into the human consciousness. As a result, people take on individual responsibilities, both to the fire and to the activities surrounding it.</p> <p>Without avenues for individualized participation, communities stagnate and break apart. That’s why Ara has prioritized enabling participation from as many different parties as possible, without sacrificing ease of use. Creators, publishers, businesses, and fans all play a key role in enabling success, and one’s technical know-how should not limit one’s ability to participate.</p> <p><strong>Creators</strong> regain control over their content, deciding for themselves how they’d like to package, prepare, and deliver it to fans. Our free, open source <a href="https://ara.one/app">Ara File Manager</a> allows anyone to become a content creator, and under whatever terms they’d like!</p> <p><strong>Publishers</strong> leverage finer-grained control over content licensing and DRM, enabling innovative new approaches to digital licensing and distribution.</p> <p><strong>Businesses</strong> are able to integrate Ara’s technology to power their content delivery, significantly reducing costs for the entire digital content ecosystem.</p> <p><strong>Fans</strong> enjoy the opportunity to be able to truly “vote with their dollars” again and contribute in a positive way to their favorite artists’ careers, without having to sit through any advertisements or have their usage tracked.</p> <h2 id="trust-and-safety">Trust and Safety</h2> <p>Early humans used their control over fire to protect themselves from dangerous predators, other hostile tribes of humans, and from the natural elements. The illuminating flame helped quickly identify newcomers during nighttime, providing a reliable means for quickly establishing trust.</p> <p>A distributed framework flourishes based upon its ability to gain trust among both current participants and potential new entrants. At Ara, we leverage blockchain technology to ensure objective, unquestionable transaction records, which can be publicly verified by anybody. What better way to check whether your favorite filmmaker received your payment for their latest movie?</p> <p>Without powerful safety mechanisms and an in-built system of trust, distributed media just feels a whole lot riskier. That’s why we built <a href="https://arablocks.github.io/guides/#/ara-identity">Ara Identity</a>, which offers a powerful, foolproof way to ensure the validity and identity of peers. This not only helps to harden our network against potential security vulnerabilities — it also eliminates the chances for identity theft, payment scamming, and other fraud which has plagued online commerce from the get-go.</p> <p>Ara’s outstanding encryption implementation means your content purchasing records are incredibly safe. We leverage the same encryption technology as the <a href="https://www.newsweek.com/bad-news-fbi-edward-snowdens-favorite-chat-app-signal-just-got-50m-funding-816035">Snowden-approved</a> Signal messaging app. Since all our encryption technology was built into the protocol from the very start, users will not need to learn any new security tricks or jump through any new hoops just to be protected. That’s a win for all parties involved, without a doubt.</p> <h2 id="innovation-through-tradition">Innovation Through Tradition</h2> <p>Sometimes, we are able to make extraordinary things happen by paying attention to the successes of the past. Even though today’s world feels seemingly detached from the campfires of our ancestors, much of what we know and trust about the world came about through their use. By applying the campfire’s virtues to the exciting new developments in distributed media, we have instilled sanity, reliability, and innovation all into one simple protocol.</p> <p>Though we may never obtain the ubiquity of fire, Ara is blazing an exciting new trail in content networks, and we invite each and every one of you to join us on our journey. Visit <a href="https://ara.one">https://ara.one</a> to learn more.</p> Audio Mastering Workflow for Linux 2019-05-05T12:15:00-04:00 https://grathwohl.me/2019/05/05/linuxmasteringflow_1 <p>This series of posts will detail the audio mastering workflow I employ on my main Linux workstation:</p> <ul> <li> <p>Watercooled Full ATX Tower</p> </li> <li> <p>Intel i9 @ 4.2Ghz</p> </li> <li> <p>64GB ECC RAM</p> </li> <li> <p>Arch Linux with the <code class="language-plaintext highlighter-rouge">linux-rt</code> kernel patches</p> </li> <li> <p><a href="https://www.rme-audio.de/en/products/hdsp_9632.php">RME Hammerfall DSP 9632</a> PCI Audio Card</p> </li> <li> <p><a href="https://babyface.rme-audio.de/">RME Babyface Pro</a> USB Audio Interface</p> </li> </ul> <h2 id="why">Why?</h2> <p>The answer is simple: <em>Linux provides the best audio engineering workflow for audio professionals - hands down.</em> Don’t believe me? I’ll prove it!</p> <h3 id="audio-engine">Audio Engine</h3> <p>Audio is a second-class citizen in every modern OS. From macOS’s progressively-worsening “CoreAudio” to Windows’ pathetic ASIO implementation, the proprietary software world has fewer reasons every day to coerce you into a non-free software environment.</p> <p><a href="http://jackaudio.org/">JACK</a> allows audio to be your OS’s main consideration.</p> <p>It lets you route <em>anything</em> to <em>any other thing</em> and does so at minimal to no latency!</p> <h3 id="extensibility-without-compromise">Extensibility Without Compromise</h3> <p>Want to add a new source connection or a track insert? Audio doesn’t stop playing and no clicks are emitted just because you wanted to do that.</p> <p>Want to use Wine to power your Windows VSTs inside your DAW? There’s an application for that.</p> <p>Want to use the <code class="language-plaintext highlighter-rouge">realtime</code> kernel patches to ensure your latency never extends beyond 1.9ms? Just <code class="language-plaintext highlighter-rouge">yay -S linux-rt</code> and go brew some coffee. Within an hour you should have a fully functioning realtime kernel OS with incredible latency features. And trust me, you <em>do</em> want to use <code class="language-plaintext highlighter-rouge">linux-rt</code>. Anyone who tells you that audio engineers won’t benefit from using realtime patches doesn’t in any way understand our plight. Using <code class="language-plaintext highlighter-rouge">linux-rt</code> I am able to achieve latencies under 2ms at 44.1kHz with 3 periods per block, using my RME Babyface Pro in Class-Compliant Mode.</p> <h2 id="how">How?</h2> <p>It actually is quite simple. However, I do have to call out a caveat:</p> <h3 id="caveat">CAVEAT</h3> <p>The system I’ve configured and which has been successful for me, is by no means an “all rounder”. By this, I mean that users must go to some lengths to ensure that no extraneous processes are running, no systemd services are operational, etc., unless <em>it is absolutely necessary</em> for your audio tasks.</p> <p>If you use <code class="language-plaintext highlighter-rouge">linux-rt</code> expecting the same stability for, say, databases and video processing, you’ll be very disappointed. To do this work, simply boot back into the stable <code class="language-plaintext highlighter-rouge">linux</code> kernel that comes with your Arch Linux installation.</p> <h3 id="getting-started">Getting Started</h3> <p>To get started, simply install a new Arch Linux base OS on your audio PC.</p> <p>Next, compile the <code class="language-plaintext highlighter-rouge">linux-rt</code> sources by installing the <code class="language-plaintext highlighter-rouge">yay</code> AUR helper, then runnning:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>yay -S linux-rt </code></pre></div></div> <p>You should also acquire the following realtime-oriented tools:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>yay -S rtapp rtirq realtime-suggestions cpupower </code></pre></div></div> <h5 id="recommended-settings">Recommended Settings</h5> <p>Enable the realtime services to run at boot:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>systemctl enable rtapp &amp;&amp; systemctl start rtapp systemctl enable rtirq &amp;&amp; systemctl start rtirq systemctl enable cpupower &amp;&amp; systemctl start cpupower </code></pre></div></div> <p>Open the <code class="language-plaintext highlighter-rouge">rtapp</code> config file at <code class="language-plaintext highlighter-rouge">/etc/rtapp/rtapp.conf</code> and add any binaries for audio programs you use there.</p> <p>Set swappiness lower than default by running:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo sysctl vm.swappiness=10 </code></pre></div></div> <p>Set the CPU governor to <code class="language-plaintext highlighter-rouge">performance</code>:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo cpupower frequency-set -g performance </code></pre></div></div> <p>Finally, run the <code class="language-plaintext highlighter-rouge">realtime-suggestions</code> program to see what it thinks about your configuration. It will give you warnings if anything should change.</p> <h3 id="software">Software</h3> <h4 id="jack2--cadence">JACK2 &amp; Cadence</h4> <p>We will be using <code class="language-plaintext highlighter-rouge">jack2</code> because using <code class="language-plaintext highlighter-rouge">jack</code> in 2019 is stupid! The multi-threading, MIDI, and other various features of <code class="language-plaintext highlighter-rouge">jack2</code> are deal-breakers, not nice-to-haves.</p> <p>Also we will use <code class="language-plaintext highlighter-rouge">cadence</code> to configure our JACK settings.</p> <p><img src="https://kx.studio/screenshots/cadence1.png" alt="" /></p> <p>Get everything you need by running:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>yay -S jack2 cadence alsa-firmware alsa-utils alsa-tools </code></pre></div></div> <h5 id="recommended-settings-1">Recommended Settings</h5> <p>For any high-performance real-time audio synthesis work, such as SuperCollider programming, it’s strongly encouraged that you run at the lowest sampling frequency possible (I use 44.1kHz and if you use anything higher, you need to <a href="https://xiph.org/video/vid2.shtml">watch this video</a>.)</p> <p>I am able to place my RME Babyface Pro at 44.1kHz with a 3-period buffer and a block latency of 64 which results in a final round-trip latency of 1.5ms!!! That’s right - on consumer hardware, I am running an audio workstation that has a round-trip of under two milli-freaking-seconds! It’s a great time to be an audio engineer.</p> <h4 id="non-daw">non-daw</h4> <p>I love <code class="language-plaintext highlighter-rouge">non-daw</code>. It’s a modular DAW which allows one to roll their own audio workflows.</p> <p><code class="language-plaintext highlighter-rouge">non</code> comes with:</p> <p><code class="language-plaintext highlighter-rouge">non-mixer</code>: A super fast audio mixing engine. It’s got an extremely functional <a href="http://non.tuxfamily.org/mixer/doc/spatialization-console.png">spatial panner</a>!</p> <p><code class="language-plaintext highlighter-rouge">non-session-manager</code>: A session management interface which handles all plugin states, audio interconnections, and routing specifications.</p> <p><code class="language-plaintext highlighter-rouge">non-sequencer</code>: A midi and audio file tracking sequencer which is serviceable for most uses as long as you’re not looking to become a Venetian Snares copycat. It’s got this novel concept of a “phrase” which is essentially a way to break down compositions into units of musical gestures.</p> <p><code class="language-plaintext highlighter-rouge">non-timeline</code>: A multi-track timeline view which enables lining up all your tracks, components, and outputs, unified by a single clock.</p> <p>For mastering purposes, I have installed the following:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>yay -S non-mixer non-session-manager </code></pre></div></div> <h4 id="carla-vst-bridge">Carla: VST Bridge</h4> <p>Carla is the software that allows us to run our Windows VST plugins (which we have <strong>PAID FOR</strong> of course) in Linux without any sacrifices in terms of quality, efficiency, or reliability.</p> <p><img src="https://camo.githubusercontent.com/20961a44e99d7a813837fa1ab5c3ab2395809428/687474703a2f2f6b7873747564696f2e6c696e7578617564696f2e6f72672f73637265656e73686f74732f6361726c612e706e67" alt="" /></p> <p>Before installing Carla, let’s prepare our system:</p> <p>First step is to make sure we are reading from Arch’s <code class="language-plaintext highlighter-rouge">multilib</code> repo. To do that, uncomment the multilib repository information in <code class="language-plaintext highlighter-rouge">/etc/pacman.conf</code>.</p> <p>Next, we install wine: <code class="language-plaintext highlighter-rouge">yay -S wine-staging</code>. It is critical that you use <code class="language-plaintext highlighter-rouge">wine-staging</code> from the AUR and not the production-ready wine binary, because only <code class="language-plaintext highlighter-rouge">wine-staging</code> supports running VST plugins in RT mode.</p> <p>Now you can install carla and all the necessary modules:</p> <p><code class="language-plaintext highlighter-rouge">yay -S carla-git carla-bridges</code></p> <p>Now, you can use Carla to load Windows VST2 and VST3s (both 32bit and 64bit supported) <a href="https://kx.studio/Applications:Carla#Usage">with ease</a>!</p> <h5 id="recommended-settings-2">Recommended Settings</h5> <p>For mastering, run Carla’s engine is in <em>continuous rack</em> mode. This allows a user to order their plugins like on a physical audio rackmount cabinet, and handles all audio routing and latency concerns for you. The tradeoff is that since this is a simplified routing approach, advanced features like infinite patchbay routing are not available to you. But for mastering purposes, this is usually irrelevant.</p> <h2 id="conclusion--up-next">Conclusion / Up Next</h2> <p>This concludes <em>Part 1</em> of this series of posts. You should now be able to run your new Arch Linux install in <code class="language-plaintext highlighter-rouge">realtime</code> mode, achieving incredible latencies without risking system stability. And you should even be able to run your Windows VSTs within your Arch system by utilizing <code class="language-plaintext highlighter-rouge">wine-staging</code>!</p> <p>In <em>Part 2</em> of this series, I’ll demonstrate how to manage your audio signal flows, custom routing, plugin states, and more. We’ll also introduce the concepts of MIDI and OSC communication with <code class="language-plaintext highlighter-rouge">jack2</code> and the ALSA framework so that you can use your interface hardware to interact with Ardour5, non-daw, etc.</p> <p>All of the musical and audio engineering work I do originates from my Linux audio systems. Check out what I’ve achieved using only free and open source software, at the <a href="https://multipli.city">Sonic Multiplicities homepage</a>. You can hear many of my mastering projects at <a href="https://soundcloud.com/sacreddata">my old company’s SoundCloud account</a>.</p> Partitipaction: Digital Media's Missing Ingredient 2018-11-15T05:44:00-05:00 https://grathwohl.me/2018/11/15/Participation_Digital_Media <iframe src="https://docs.google.com/presentation/d/e/2PACX-1vQArwmb8OTVs-eBaUDXaj7XgIPdAkGPvBGYuW5nQWezFNsdeP-alDbX9cvaQRp5UoiQ3p4xSVMT47sm/embed?start=true&amp;loop=false&amp;delayms=30000" frameborder="0" width="720" height="480" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe> Playing Your Audible Audiobooks In Linux 2018-09-10T06:44:00-04:00 https://grathwohl.me/2018/09/10/audibleffmpeg <p>If you want to convert an Audible .aax file into a less restrictive file format, you can do so with ease using <a href="https://ffmpeg.org/">FFmpeg</a>, one of my all-time favorite GPL projects.</p> <p>Navigate to the directory where the .aax file is that you’d like to test on, and perform the following on the command line:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffmpeg -activation_bytes 1CEB00DA -i test.aax -vn -c:a copy output.mp4 </code></pre></div></div> <p>Doing this creates an mp4 container with only an AAC audio stream inside. By adding the <code class="language-plaintext highlighter-rouge">-c:a copy</code> argument, we’re telling FFmpeg to do a stream copy of the audio, negating the need to compress and process the audio again. Since Audible audio is encoded at 64kbps HE-AAC or lower, this argument is essential to retain quality!</p> <p>This MP4 can now be played on any conventional audiovisual playback software that supports the MP4 container file. Enjoy!</p> Hacking Isn’t Searching; It’s Quartering 2018-06-25T08:00:00-04:00 https://grathwohl.me/2018/06/25/Hacking-Is-Quartering <p>A common critique of the deep state’s dragnet surveillance operation is to depict it as a violation of the Fourth Amendment. The pundits and politicians who publicly railed against the recent NSA and FBI revelations commonly referred to the issue as one of “warrant-less” wiretapping, data collection, and spying.</p> <p>But that isn’t quite the game that signals intelligence analysts and network security experts are playing.</p> <p>The invocation of the term “warrant-less” is unsurprising; it hearkens back to the Watergate scandal, and by making that comparison, it helps to make the case that these surveillance programs were just as serious and important as that major kerfuffle back in 1972.</p> <p>The issue with this is that bugging a hotel room and tampering with evidence are such benign and irrelevant acts compared with what the NSA does routinely. In reality, recent dragnet surveillance revelations are far more destructive, wide-ranging, and horrifying than anything Richard Nixon was capable of doing. The NSA does not fuck around.</p> <p>Though a well-intended PR trick, the term “warrant-less” does more to downplay the seriousness of governmental dragnet surveillance than it wakes up the general populace. Likening the NSA’s work to merely a search or seizure of evidence significantly misinforms the public about the nature of hacking in a number of ways. Let’s pick them apart, shall we?</p> <h3 id="the-nsa-plays-offense-not-defense">The NSA Plays Offense, Not Defense</h3> <p>Perhaps most important to remember is that the NSA is part of the Department of Defense; it is a military organization, not law enforcement.</p> <p>In almost every instance, hacking a network or device is not an act of defense, but rather an act of offense. When done by the NSA, you can expect it to be a particularly aggressive act — not one to conduct law enforcement activities, but to steal secrets, or to weaponize computing and network hardware.</p> <p>When a three-letter agency breaks into the system of the Average Joe, that person is usually not the target. Their system was merely a means to some other more dastardly end. For example, a compromised system is often leveraged by the attacker against their agency’s final target, which is often some other state actor. Worse yet, when these attacks are carried out, there’s no guarantee that the agency has masked your private information from the device. So if the target on the other end is capable, they may “hack back,” leaving ordinary citizens right in the crossfire.</p> <h3 id="theres-no-warrant-for-committing-atrocities">There’s No Warrant for Committing Atrocities</h3> <p>Once they gain access to your machine, there are many things that the NSA’s goons can do, and almost all of them are illegal to do in any circumstance no matter who you are, warrant or otherwise. These continued violations of our Constitution’s fundamental protections resembles nothing short of a human rights atrocity.</p> <p>Some different types of these egregious attacks include:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Denial of Service: Overloading a device or network with falsified activity, for the purpose of bringing down critical functionality. This could be as inconsequential as a kid pulling the fire alarm at their middle school, or could be as deadly as an aerial assault by a bomber jet. It all depends on the infrastructure that was attacked and the users impacted by the attack. Either way, it is a crime that cannot be excused by a warrant from a judge. Subversion: Involves the manipulation or degradation of data, in order to incite lowered confidence in the integrity of the information the network provides. This is ultimately a form of PSYOPS — psychological warfare. On the one hand, we routinely criticize Russia for engaging in this kind of activity against the US. On the other hand, we conceive of such warfare as a form of cruel and unusual punishment when it is used against American citizens. This dual narrative could even be seen as the NSA engaging in an ongoing subversion attack against the American people. Masquerade: When an attacker elevates their own permissions/privileges within a computer system in order to leverage credentials to which they should not have access. Imagine if a person broke into the local police headquarters, armed themselves in official gear and a police badge, and then used that false authority to intimidate their enemies. I would hope that no judge on this planet would grant a warrant for such an activity. Forgery: When an attacker dispatches messages from a computer network, pretending to be somebody they aren’t. This is the kind of attack that would happen if a Subversion and a Masquerade had a baby. It involves masquerading as a different computer user in order to subvert the computer system’s messaging systems. Forgery is a state and federal crime in all US jurisdictions. </code></pre></div></div> <h3 id="could-the-constitution-still-be-helpful">Could the Constitution Still Be Helpful?</h3> <p>So no, the Fourth Amendment doesn’t have a whole lot to say about the NSA’s activities. Luckily, unlike most of our current elected officials, the Founding Fathers were not idiots. The Bill of Rights still can be of major assistance in the fight against the offensive cyber operations of our military. We just need to rethink which amendment to focus upon.</p> <p>There is only one amendment to the Constitution that addresses the relationship between the citizenry and the military — the Third Amendment. And it is stunningly apt in helping to make sense of exactly why the NSA’s actions are so objectionable.</p> <p>The Third Amendment reads as follows, with my own emphases:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>No Soldier shall, in time of peace be quartered in any house, without the consent of the Owner, nor in time of war, but in a manner to be prescribed by law. </code></pre></div></div> <h3 id="citizens-1-military-0">Citizens: 1. Military: 0.</h3> <p>As an American, your private property rights as they pertain to the military are quite clear: your rights are superior to theirs. During peace time, the federal government’s soldiers cannot commandeer, lodge themselves into, or otherwise tamper with your property just to carry out their missions.</p> <p>In a time of war Congress would need to vote to approve explicitly what kind of quartering must be allowed by citizens, and that hasn’t happened yet. This is moot anyway, because we aren’t in a time of war, nor have we been since WWII.</p> <h3 id="hey-dummy-it-says-house-not-computer">Hey Dummy! It Says “House,” not “Computer”</h3> <p>Some may object to using the term “house” to describe a computer. But it is certainly reasonable to consider a computer under this definition. These days, many individuals have more of their lives inside of their personal computing devices — their PCs, smartphones, and Fitbits — than inside their actual domiciles. Modernity has even brought us to a point where the concept of the digital nomad is an actual thing.</p> <p>Still not convinced?</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Where do you read the majority of your news — online, or in the newspaper? Where do you have more photos stored — on your smartphone, or in physical photo albums? Where are the majority of your 2017 financial documents and records stored — a computer, or a filing cabinet? </code></pre></div></div> <p>Exactly.</p> <h3 id="this-is-an-important-amendment-that-most-people-forget-about">This is an Important Amendment that Most People Forget About</h3> <p>The Third Amendment was briefly debated before the ratification of the US Constitution, because the consensus was broad: we had to make sure that the Quartering Acts could not happen again. They were, after all, one of the primary tensions that led to the American Revolution in the first place. The Founders knew that the forceful quartering of an occupying force was no burden to force onto a free citizenry.</p> <p>Perhaps this is why the Third Amendment is so stunningly broad in its definition. Its text begins by placing a restriction on the soldier, so the Third Amendment grants not a positive right per se, but rather a negative right — one that explicitly restricts the actions that may be taken by a soldier of the US. It then goes on to state that it protects the rights of the owner of any house, which means that the house need not even exist on American soil to be protected.</p> <h3 id="what-now">What Now?</h3> <p>What we are trying to stop here is not the mere digital canoodling of the local police or even the FBI. This isn’t an effort to thwart the CIA’s numerous cyber warfare tools exposed by WikiLeaks. What is at risk by not taking appropriate action with respect to the government’s violation of the Third Amendment is the continued sanctioning of their actions by the world’s citizens and, more troubling, the corporations they patronize.</p> <p>At the end of the day, the Third Amendment is a federal law that establishes behavioral requirements for the United States military — the very same group committing by far the most atrocious cyber offensives throughout the world. All because lawmakers, politicians, and yes, statistically even you — don’t understand a damn thing about cybersecurity and the evolving digital war frontier.</p> <p>What’s worse is that people know just as little about the way American government and law is designed to work, so most digital citizens are unprepared to grasp this proverbial double-edged sword from either end. That needs to change immediately. The best takeaway I could ask a layperson to have after considering this is that the ability to intelligently communicate about Constitutional law is something that computer experts and computer users alike must begin to take seriously. What we do for a living is not child’s play — even a child playing on an iPad is no longer mere child’s play. We all have something significant to lose if we lose our fight for our digital rights against the state.</p> <p>We could try to blame our broken education system for our citizens’ inability to grasp basic American civics. However, I’m sure most Americans agree by now that this once great and meaningful aspect to American life — our public education system — is a lost cause beyond any kind of meaningful repair.</p> <h3 id="dont-consent">Don’t Consent!!!</h3> <p>I don’t have any truly good advice about where to go from here. I’m not a lawyer, so nothing I say will be even close to helpful on a legal front. So, my best advice is simply: don’t consent.</p> <p>Make it abundantly, explicitly clear to any intruder into your network that you do not consent to their illegal activities, and that you would like them to exit your property immediately. This won’t help protect you against anything, but maybe if enough people start to do this, it will catch on, and then somebody with actual legal expertise could inform us on how to best proceed in getting our Third Amendment protections back in some legislative fashion.</p> <p>You can signal your lack of consent to an attacker by learning how to set your computer’s login “message of the day” — aka your /etc/motd file. Better yet, donate to the Electronic Frontier Foundation and get yourself some of these lovely stickers to proclaim your distrust of government surveillance in both the physical and digital realms.</p> FFmpeg 3.0 Is Here 2016-02-14T23:11:00-05:00 https://grathwohl.me/2016/02/14/ffmpeg3ishere <p>The team behind FFmpeg unveiled a major new release, “Einstein,” recently, and in the release notes appears a strongly-worded suggestion:</p> <p><code class="language-plaintext highlighter-rouge">We strongly recommend users, distributors, and system integrators to upgrade unless they use current git master.</code></p> <p>(By the way, if you’re thinking that today may be the day to look into FFmpeg’s git master – don’t. Seriously, just don’t. This is no slight against the FFmpeg team - it’s just that with a project of this nature, you have to build the nightly release every day to get helpful suppo rt. Developers are committing small patches to individual subsets of the FFmpeg project every day, which is why the release candidates are a sure bet for most folks.)</p> <p>There are many of you out there that run FFmpeg in production, so the team’s urgency to get users to update should be taken as an indication that big changes have come. Here’s some noteworthy entries in 3.0’s <a href="https://git.videolan.org/?p=ffmpeg.git;a=blob_plain;f=Changelog ;hb=n3.0">release notes</a>:&lt;/p&gt;</p> <div class="highlighter-rouge"><pre class="highlight"><code># This feature will allow content enterprises to create encrypted proxies of their sensitive content - Common Encryption (CENC) MP4 encoding and decoding support # The DXV codec is used in live visual performances (often referred to as VJing) - DXV decoding # Stereo audio processing tools like these hint at optimizing audio tracks for headphone listeners - extrastereo filter - stereowiden filter - stereotools filter # FFmpeg's AAC improvements mean fewer patent/royalty issues when creating canonical MP4s - extensive native AAC encoder improvements and removal of experimental flag # AcousticID generation for audio tracks! - chromaprint fingerprinting muxer </code></pre> </div> <p>Sheesh! And that’s not even the half of it. Suffice to say, this FFmpeg release is a significant one. Ahead, I will explore the above-listed features in detail.</p> <h2 id="common-encryption-support-for-mp4">Common Encryption support for MP4</h2> <p>This feature is not documented on FFmpeg.org’s <code class="highlighter-rouge">ffmpeg-all.html</code> page. After a quick trip to #ffmpeg on freenode, I discovered that no documentation currently exists for this feature except for on the command line:</p> <div class="highlighter-rouge"><pre class="highlight"><code>$ ffmpeg -h muxer=mov | grep -encryption_scheme # -encryption_scheme &lt;string&gt; E....... Configures the encryption scheme, allowed values are none, cenc-aes-ctr </code></pre> </div> <p>Though a helpful start, I have not yet gotten this feature to work.</p> <h2 id="chromaprint-support">Chromaprint Support</h2> <p>Chromaprint’s AcousticID is an audio fingerprinting library. It’s pretty cool that they’ve implemented support for this within FFmpeg! There are some caveats, however. To generate an AcousticID of an audio file in FFmpeg, that file must be a <code class="highlighter-rouge">single signed native-endian 16-bit raw audio stream</code>. Yikes!</p> <h2 id="native-aac-codec">Native AAC Codec</h2> <p>Word on the street was, at first, that the FFmpeg native aac codec is now outperforming the fdk-aac codec. I was dubious of this claim, and recently, a respected super mega-nerd over at Hydrogenaud.io released <a href="https://hydrogenaud.io/index.php/topic,111085/topicseen.html">his own thoughts</a> on the newly-improved codec. Needless to say, <em>if you are updating to 3.0, do not get rid of libfdk-aac!</em> The fdk-aac codec is still the only available FFmpeg aac library that truly supports VBR, and it still wins in the sound quality department.</p>