meain/blogmeain's blog2024-02-08T00:00:00Zhttps://meain.io/Abin Simonmail@meain.ioHow to make Vim and Tmux friends with system clipboard2017-05-27T00:00:00Zhttps://blog.meain.io/2017/make-vim-and-tmux-friends-with-system-clipboard/<p>Fist of all, a small apology. The <code>tmux</code> solution is a bit specific and works only if you are using <code>macOS</code> and <code>iTerm</code> as your terminal emulator.</p>
<h1 id="get-vim-to-play-nice" tabindex="-1">Get <code>Vim</code> to play nice <a class="direct-link" href="https://blog.meain.io/2017/make-vim-and-tmux-friends-with-system-clipboard/#get-vim-to-play-nice">#</a></h1>
<p>Getting <code>Vim</code> working with the system clipboard is the easy part.</p>
<h2 id="on-macos-(-sierra-)" tabindex="-1">On macOS ( Sierra ) <a class="direct-link" href="https://blog.meain.io/2017/make-vim-and-tmux-friends-with-system-clipboard/#on-macos-(-sierra-)">#</a></h2>
<p>This is very easy. You will have clipboard support out of the box.
You can get the clipboard contents by using</p>
<pre><code>"*p
</code></pre>
<p>for paste and</p>
<pre><code>"*y
</code></pre>
<p>for copy ( yank )</p>
<h2 id="on-linux" tabindex="-1">On Linux <a class="direct-link" href="https://blog.meain.io/2017/make-vim-and-tmux-friends-with-system-clipboard/#on-linux">#</a></h2>
<p>Well, its a bit harder on Linux, not that much though.
First you will have to install a clipboard manager ( I am not sure its called that ) like <a href="https://github.com/astrand/xclip"><code>xclip</code></a>.
Now, once that's down you can use</p>
<pre><code>"+p
</code></pre>
<p>for paste and</p>
<pre><code>"+y
</code></pre>
<p>for copy ( yank )</p>
<h1 id="get-tmux-to-play-nice" tabindex="-1">Get <code>Tmux</code> to play nice <a class="direct-link" href="https://blog.meain.io/2017/make-vim-and-tmux-friends-with-system-clipboard/#get-tmux-to-play-nice">#</a></h1>
<p>This is the hard part and I am really sorry that I haven't got time to play around with it on Linux.</p>
<h2 id="set-up-iterm" tabindex="-1">Set up iTerm <a class="direct-link" href="https://blog.meain.io/2017/make-vim-and-tmux-friends-with-system-clipboard/#set-up-iterm">#</a></h2>
<p>This solution is specific to <a href="https://www.iterm2.com/">iTerm</a>, so download and install it first.
Now enable <code>Applications in terminal may access clipboard</code> option from iTerm preferences.</p>
<p><img src="https://i.imgur.com/wo5c6Ev.png" alt="screenshot-iterm-pref" /></p>
<h2 id="set-up-tmux" tabindex="-1">Set up Tmux <a class="direct-link" href="https://blog.meain.io/2017/make-vim-and-tmux-friends-with-system-clipboard/#set-up-tmux">#</a></h2>
<p>Now we have to set up Tmux.</p>
<p>First step is to install <code>reattach-to-user-namespace</code> using brew:</p>
<pre class="language-shell"><code class="language-shell">brew <span class="token function">install</span> reattach-to-user-namespace</code></pre>
<p>Now add this line to your tmux config file at <code>~/.tmux.conf</code></p>
<pre><code>set-option -g default-command "reattach-to-user-namespace -l bash"
</code></pre>
<p>Replace <code>bash</code> at the end with your shell. For example if your shell is <code>zsh</code> do:</p>
<pre><code>set-option -g default-command "reattach-to-user-namespace -l zsh"
</code></pre>
<p>Now you can set <code>tmux</code> to use <code>vim</code> keys for copy and stuff.
Just add the below lines to your <code>tmux</code> config file at <code>~/.tmux.conf</code></p>
<pre><code>set-window-option -g mode-keys vi
if-shell "test '\( #{$TMUX_VERSION_MAJOR} -eq 2 -a #{$TMUX_VERSION_MINOR} -ge 4 \)'" 'bind-key -Tcopy-mode-vi v send -X begin-selection; bind-key -Tcopy-mode-vi y send -X copy-selection-and-cancel'
if-shell '\( #{$TMUX_VERSION_MAJOR} -eq 2 -a #{$TMUX_VERSION_MINOR} -lt 4\) -o #{$TMUX_VERSION_MAJOR} -le 1' 'bind-key -t vi-copy v begin-selection; bind-key -t vi-copy y copy-selection'
</code></pre>
<h3 id="and-you-are-good-to-go!" tabindex="-1">And you are good to go! <a class="direct-link" href="https://blog.meain.io/2017/make-vim-and-tmux-friends-with-system-clipboard/#and-you-are-good-to-go!">#</a></h3>
<h1 id="and-a-bonus-tip" tabindex="-1">And a bonus tip <a class="direct-link" href="https://blog.meain.io/2017/make-vim-and-tmux-friends-with-system-clipboard/#and-a-bonus-tip">#</a></h1>
<p>Want to get mouse scrolling on tmux? Add the following to your <code>tmux</code> config.</p>
<pre><code>bind -n WheelUpPane if-shell -F -t = "#{mouse_any_flag}" "send-keys -M" "if -Ft= '#{pane_in_mode}' 'send-keys -M' 'copy-mode -e'"
</code></pre>
<p>This will, as soon as you start scrolling get you into copy mode. You can even select text inside tmux and it will copy as soon you have completed selecting into your clipboard.</p>
How Emacs took over my Vim life2017-06-13T00:00:00Zhttps://blog.meain.io/2017/how-emacs-took-over-my-vim-life/<blockquote>
<p>Edit: I am back to using <code>neovim</code>.</p>
</blockquote>
<p>I was a fanatic Vim user for about two years and use to believe that Vim is the only text editor that was <code>cool</code>. But, now I use <code>Emacs</code>.</p>
<blockquote>
<p>My <a href="https://github.com/meain/dotfiles/tree/master/emacs">emacs</a> and <a href="https://github.com/meain/dotfiles/blob/master/nvim/.config/nvim/init.vim">neovim</a> config</p>
</blockquote>
<h1 id="what-is-the-problem-with-vim%3F" tabindex="-1">What is the problem with <code>Vim</code>? <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#what-is-the-problem-with-vim%3F">#</a></h1>
<p>Well, Vim was a great tool for editing <code>text</code>, not for editing code. The one main issue I was facing with Vim was due to its synchronous nature of plugins. It really got show when I had quite a bit of stuff going on. Yeah I know about <code>Neovim</code> and now with Vim 8 we have got async processing but its not on par with <code>Emacs</code> on that. We have <a href="https://github.com/w0rp/ale">ale</a> and <a href="https://github.com/Shougo/deoplete.nvim">deoplete</a> which are awesome but it lot of other things was messy in the plugin landscape.</p>
<p>The main issue with Vim was if I had any large file opened in Vim when I have a long lis of plugins( which I do ) I was having a really hard time getting anything done. Kudos to <a href="https://www.sublimetext.com/">Sublime Text</a> on handling that so well. This was once and issue when I had a big LaTeX file which I could <strong>not</strong> edit in Vim because it was too slow after adding the LaTeX plugin.</p>
<p>Now, another reason for switching from Vim is due to its lack of good gui support. Yeah we have gvim, macvim ect but those don't provide anything more than more colors. <code>Emacs</code> could do images inline and that was really something for me.</p>
<p>Now comes the biggest pain with using Vim was when I had to work with <code>Vue</code>. Vim has a really hard time dealing with templates. You have <a href="https://github.com/posva/vim-vue">Vue plugin</a> in Vim but its not that good. You lack on good autocompletion and commenting. I used to use tpope's <a href="https://github.com/tpope/vim-commentary">commentary</a> but I could not use that in Vue files and had to switch to <a href="https://github.com/scrooloose/nerdcommenter">NERD Commenter</a> and do <a href="https://github.com/posva/vim-vue#how-can-i-use-nerdcommenter-in-vue-files">this</a> to get the commenting working and still it was a slow and got the comment type wrong at times for some reason.</p>
<h1 id="what-is-there-to-gain-with-emacs%3F" tabindex="-1">What is there to gain with Emacs? <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#what-is-there-to-gain-with-emacs%3F">#</a></h1>
<p><strong>Oh you bet there is</strong></p>
<p>One of the best plugins you will ever see is <a href="https://github.com/magit/magit">Magit</a> and <code>Emacs</code> is home to it. I used to use <code>Emacs</code> just to use <code>Magit</code> even when I was a Vim use due to its ability to do chunked commits.</p>
<p>Well, lets get to more obvious benefits.</p>
<p><code>Elisp</code> though more daunting at first is a much better and powerful language than <code>Vimscript</code>. You will learn to love elisp and its self documenting nature.</p>
<p>You gain a lot of things GUI. Like inline images, different fon't sizes in a buffer etc.</p>
<p>Also the whole community arround <code>Emacs</code> is different from that of <code>Vim</code>. In <code>Emacs</code> plugins work together rather than messing up the other one. You can do that with <code>Vimscript</code> but nobody actually does that. <code>Elisp</code> feels more like a proper language and everybody plays nicely with each other.</p>
<p>The biggerst gain for me was the async nature of <code>Emacs</code>. That in itself was a big win for me.</p>
<h1 id="what-you-won't-lose%3F" tabindex="-1">What you won't lose? <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#what-you-won't-lose%3F">#</a></h1>
<p>Well, you won't lose most of the core Vim stuff. You have <a href="https://github.com/emacs-evil/evil">Evil mode</a>. It is <code>Vim</code> emulation layer on top of <code>Emacs</code> and is really good. It emulates almost everything that <code>Vim</code> does. I have not had any issues with the emulation but just saying <em>almost everything</em> to be on the safe side.</p>
<p>Another thing you won't lose is <a href="https://github.com/tpope">Tim Pope's</a> plugins. Everybody loves @tpope and even people in <code>Emacs</code> community and we have similar to plugins to <code>commentary</code>, <code>surround</code> and others. You will feel right at home with it.</p>
<h1 id="what-you-will-lose%3F" tabindex="-1">What you will lose? <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#what-you-will-lose%3F">#</a></h1>
<p>Well, you ought to lose something right? It might not be a big deal to some but you kinda lose the whole unixy feel. With <code>Vim</code> the whole philosophy is to provide a good editing environment in <code>Vim</code> and hand over other things to other programs. One main example for this is to sort in vim you select a region of text and call the sort unix function on that. Maybe its just me, but the <code>Emacs</code> community felt a bit different and wanted everything inside <code>Emacs</code>. Not that it is a bad experience or that <code>Emacs</code> can't pipe out to shell and do stuff. You don't feel like doing it. Apart from that I don't have felt like I lost a whole lot.</p>
<h1 id="any-other-editor-you-would-suggest%3F" tabindex="-1">Any other editor you would suggest? <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#any-other-editor-you-would-suggest%3F">#</a></h1>
<p>Well, <code>Emacs</code> and <code>Vim</code> might not be the best editor for everyone. If you are not really sold into customizing your editor a lot and wan't something that works awesome out of the box go with <a href="https://code.visualstudio.com/">VS Code</a>. Microsoft really has done a lot of work to make it awesome.</p>
<h1 id="how-hard-is-the-switch%3F" tabindex="-1">How hard is the switch? <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#how-hard-is-the-switch%3F">#</a></h1>
<p>Well it is not the smoothest ride ever. It is bit hard in the begining. You have to learn a bit of <code>elisp</code> which by the way is a great language. Also you need to go hunt for the plugins you wan't to use in the begining which is kinda fun. Well another good thing here is that in the case of <code>Emacs</code> we don't have a lot of plugins that do the same thing, but one very mature plugin and some other which are a fair bit different. Well, don't worry its not that hard.</p>
<h1 id="can-i-get-......-plugin-in-emacs%3F" tabindex="-1">Can I get ...... plugin in Emacs? <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#can-i-get-......-plugin-in-emacs%3F">#</a></h1>
<p>Oh you can, don't worry!</p>
<h1 id="lets-switch!" tabindex="-1">Lets Switch! <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#lets-switch!">#</a></h1>
<h2 id="install-emacs" tabindex="-1">Install <code>Emacs</code> <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#install-emacs">#</a></h2>
<p>Well, just install <code>Emacs</code> using <code>homebrew</code>, <code>apt</code> or <code>pacman</code> or whatever that you have. They do also have a windows binary. It not like you need to compile with python or anything like that in <code>Vim</code></p>
<h2 id="learn-some-basic-emacs" tabindex="-1">Learn some basic <code>Emacs</code> <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#learn-some-basic-emacs">#</a></h2>
<p>Well, <code>Evil</code> is great but there are some amount of <code>Emacs</code> you have to learn. You can actually remap all of this later but a knowing this in the start is good.</p>
<p>As a momento to fact that you don't have to learn a lot of <code>Emacs</code> I would like to say that I still don't know how to save a file in <code>Emacs</code>. That is how good the emulation is.</p>
<p>Well, here are the thing you have to learn:</p>
<ul>
<li><code>M-x</code> - it like <code>cmd-shift-p</code> in <code>Sublime</code> or <code>Atom</code> or <code>VSCode</code></li>
<li><code>C-g</code> - if you are in anything and you can't exit just use this</li>
<li><code>C-x C-c</code> - quit <code>Emacs</code></li>
</ul>
<p>That's it!</p>
<h2 id="how-to-get-help%3F" tabindex="-1">How to get help? <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#how-to-get-help%3F">#</a></h2>
<p><code>Emacs</code> and <code>elisp</code> is really serious about documentation and you have a really good help system.
Here are a few ways you can get help</p>
<p>These are the commands you can type to get help. Type these after you typed <code>M-x</code>.</p>
<ul>
<li><code>describe-function</code> - shows the functions's docstring, the file it is in etc.</li>
<li><code>describe-variable</code> - shows data about a variable and its current value</li>
<li><code>describe-key</code> - shows what the key you press is bound to</li>
</ul>
<p>These are the main ones that you will use. You also have commands to describe <code>major-mode</code>, <code>minor-mode</code>, <code>theme</code>, <code>symbol</code> and the list goes on and on.</p>
<p>And at the end of the day you will always have the Emacs community and Stack Overflow.</p>
<h2 id="woah%2C-modes%3F%3F-what%3F%3F" tabindex="-1">Woah, <code>modes</code>?? What?? <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#woah%2C-modes%3F%3F-what%3F%3F">#</a></h2>
<p>Well, <code>modes</code> are essentially modes. Not the best explanation I guess.
<code>Emacs</code> has two kinds of modes. <code>major</code> mode and <code>minor</code> mode.
<code>major</code> mode in <code>Emacs</code> is closest to <code>filetype</code> in Vim. So for example you have <code>python-mode</code>, <code>js-mode</code> etc.
<code>minor</code> mode in <code>Emacs</code> is the other plugins you load on top of that. For example a plugin to do commenting in <code>python-mode</code> is a minor mode.
Well, that is pretty much what modes are.</p>
<h2 id="setting-up-evil-mode" tabindex="-1">Setting up <code>Evil mode</code> <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#setting-up-evil-mode">#</a></h2>
<p>This is a good starting point. Add the following to your <code>~/.emacs</code> file.
And you have a simple <code>Evil mode</code> setup.</p>
<pre class="language-lisp"><code class="language-lisp"><br /><span class="token punctuation">(</span><span class="token keyword">require</span> <span class="token quoted-symbol variable symbol">'package</span><span class="token punctuation">)</span><br /><br /><span class="token comment">; List the packages you want</span><br /><span class="token punctuation">(</span><span class="token keyword">setq</span> package-list <span class="token punctuation">'(</span><span class="token car">evil</span><br /> evil-leader<span class="token punctuation">)</span><span class="token punctuation">)</span><br /><br /><span class="token comment">; Add Melpa as the default Emacs Package repository</span><br /><span class="token comment">; only contains a very limited number of packages</span><br /><span class="token punctuation">(</span><span class="token car">add-to-list</span> <span class="token quoted-symbol variable symbol">'package-archives</span><br /> <span class="token punctuation">'(</span><span class="token string">"melpa"</span> <span class="token punctuation">.</span> <span class="token string">"http://melpa.milkbox.net/packages/"</span><span class="token punctuation">)</span> <span class="token boolean">t</span><span class="token punctuation">)</span><br /><br /><span class="token comment">; Activate all the packages (in particular autoloads)</span><br /><span class="token punctuation">(</span><span class="token car">package-initialize</span><span class="token punctuation">)</span><br /><br /><span class="token comment">; Update your local package index</span><br /><span class="token punctuation">(</span><span class="token keyword">unless</span> package-archive-contents<br /> <span class="token punctuation">(</span><span class="token car">package-refresh-contents</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><br /><span class="token comment">; Install all missing packages</span><br /><span class="token punctuation">(</span><span class="token car">dolist</span> <span class="token punctuation">(</span><span class="token car">package</span> package-list<span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">unless</span> <span class="token punctuation">(</span><span class="token car">package-installed-p</span> package<span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">package-install</span> package<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><br /><span class="token punctuation">(</span><span class="token keyword">require</span> <span class="token quoted-symbol variable symbol">'evil</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">evil-mode</span> <span class="token boolean">t</span><span class="token punctuation">)</span><br /><br /><span class="token punctuation">(</span><span class="token keyword">require</span> <span class="token quoted-symbol variable symbol">'evil-leader</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">global-evil-leader-mode</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">evil-leader/set-leader</span> <span class="token string">"<SPC>"</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">evil-leader/set-key</span><br /> <span class="token string">"b"</span> <span class="token quoted-symbol variable symbol">'switch-to-buffer</span><br /> <span class="token string">"w"</span> <span class="token quoted-symbol variable symbol">'save-buffer</span><span class="token punctuation">)</span></code></pre>
<p>What this does is the next time you start <code>Emacs</code> it download the packages <code>evil-mode</code> and <code>evil-leader</code> (it is a plugin that helps you emulate the leader key functionality in emacs) and sets you two bindings for you.</p>
<ul>
<li><code><leader>b</code> - open a buffer list so that you can switch to a different one</li>
<li><code><leader>w</code> - save-file</li>
</ul>
<p>Oh, btw leader key is mapped to <code>space</code>.</p>
<h2 id="fixing-some-rough-edges" tabindex="-1">Fixing some rough edges <a class="direct-link" href="https://blog.meain.io/2017/how-emacs-took-over-my-vim-life/#fixing-some-rough-edges">#</a></h2>
<p>Not everything in emacs will be useful with <code>Evil mode</code> and you might disable it in some but it is great to have the same key letting you to jump from one buffer to another. So remap the C-hjkl keys to do buffer switching globally.</p>
<pre class="language-lisp"><code class="language-lisp"><span class="token punctuation">(</span><span class="token car">define-key</span> global-map <span class="token punctuation">(</span><span class="token car">kbd</span> <span class="token string">"C-h"</span><span class="token punctuation">)</span> `windmove-left<span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">define-key</span> global-map <span class="token punctuation">(</span><span class="token car">kbd</span> <span class="token string">"C-j"</span><span class="token punctuation">)</span> `windmove-down<span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">define-key</span> global-map <span class="token punctuation">(</span><span class="token car">kbd</span> <span class="token string">"C-k"</span><span class="token punctuation">)</span> `windmove-up<span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">define-key</span> global-map <span class="token punctuation">(</span><span class="token car">kbd</span> <span class="token string">"C-l"</span><span class="token punctuation">)</span> `windmove-right<span class="token punctuation">)</span></code></pre>
<p>As <code>Vim</code> users we wan't <code>ESC</code> to quit out of everything. And here is how you do it.</p>
<pre class="language-lisp"><code class="language-lisp"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">minibuffer-keyboard-quit</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span><br /> <span class="token string">"Abort recursive edit.<br /> In Delete Selection mode, if the mark is active, just deactivate it;<br /> then it takes a second \\[keyboard-quit] to abort the minibuffer."</span><br /> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">and</span> delete-selection-mode transient-mark-mode mark-active<span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">setq</span> deactivate-mark <span class="token boolean">t</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">when</span> <span class="token punctuation">(</span><span class="token car">get-buffer</span> <span class="token string">"*Completions*"</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">delete-windows-on</span> <span class="token string">"*Completions*"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">abort-recursive-edit</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">define-key</span> evil-normal-state-map <span class="token punctuation">[</span>escape<span class="token punctuation">]</span> <span class="token quoted-symbol variable symbol">'keyboard-quit</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">define-key</span> evil-visual-state-map <span class="token punctuation">[</span>escape<span class="token punctuation">]</span> <span class="token quoted-symbol variable symbol">'keyboard-quit</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">define-key</span> minibuffer-local-map <span class="token punctuation">[</span>escape<span class="token punctuation">]</span> <span class="token quoted-symbol variable symbol">'minibuffer-keyboard-quit</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">define-key</span> minibuffer-local-ns-map <span class="token punctuation">[</span>escape<span class="token punctuation">]</span> <span class="token quoted-symbol variable symbol">'minibuffer-keyboard-quit</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">define-key</span> minibuffer-local-completion-map <span class="token punctuation">[</span>escape<span class="token punctuation">]</span> <span class="token quoted-symbol variable symbol">'minibuffer-keyboard-quit</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">define-key</span> minibuffer-local-must-match-map <span class="token punctuation">[</span>escape<span class="token punctuation">]</span> <span class="token quoted-symbol variable symbol">'minibuffer-keyboard-quit</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">define-key</span> minibuffer-local-isearch-map <span class="token punctuation">[</span>escape<span class="token punctuation">]</span> <span class="token quoted-symbol variable symbol">'minibuffer-keyboard-quit</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token car">global-set-key</span> <span class="token punctuation">[</span>escape<span class="token punctuation">]</span> <span class="token quoted-symbol variable symbol">'evil-exit-emacs-state</span><span class="token punctuation">)</span><br /></code></pre>
<p>In <code>Emacs</code> you often get many prompts asking you for yes or no. Make then y or n</p>
<pre class="language-lisp"><code class="language-lisp"><span class="token punctuation">(</span><span class="token car">fset</span> <span class="token quoted-symbol variable symbol">'yes-or-no-p</span> <span class="token quoted-symbol variable symbol">'y-or-n-p</span><span class="token punctuation">)</span></code></pre>
<p>With this you are pretty much good to go with <code>Evil mode</code>. Now its all about finding the right plugins and using them.</p>
<p><em>Good luck! :)</em>
Ping me if you need help. I will be glad to help you out.</p>
Ehh, Webassembly?2017-08-27T00:00:00Zhttps://blog.meain.io/2017/ehh-webassembly/<h1 id="well%2C-hey.-so%2C-what-is-webassembly%3F" tabindex="-1">Well, hey. So, what is Webassembly? <a class="direct-link" href="https://blog.meain.io/2017/ehh-webassembly/#well%2C-hey.-so%2C-what-is-webassembly%3F">#</a></h1>
<p>To be blunt and simple it is just <code>assembly</code> on the <code>web</code>. What you do is write in a low level language such as <code>C</code>, <code>C++</code> or <code>rust</code>(a new language by Mozilla) and convert them into assembly(sort of) language and ship that to browser.</p>
<h1 id="why-new-stuff-though%3F" tabindex="-1">Why new stuff though? <a class="direct-link" href="https://blog.meain.io/2017/ehh-webassembly/#why-new-stuff-though%3F">#</a></h1>
<p>Well, JS was a language which had performance as an afterthought. It was a language that concentrated on simplicity and speed of programming. We have come a long way and made JS much better with <code>JIT</code> and maintaining a high level of WTF's per minute. But even so, it is still cannot match the speeds of lower level languages. The two main reasons for this are the <code>event loop</code>(which is actually something I kinda love for the whole idea of it) and dynamic typing(Oh boy is it dynamic).</p>
<blockquote>
<p>Just to keep this clear, <code>Webassembly</code> is not a new language</p>
</blockquote>
<h1 id="ok%2C-so-what-is-this%2C-how-do-i-get-started%3F" tabindex="-1">OK, so what is this, how do I get started? <a class="direct-link" href="https://blog.meain.io/2017/ehh-webassembly/#ok%2C-so-what-is-this%2C-how-do-i-get-started%3F">#</a></h1>
<p>Well, here is some good news and bad news depending on who you are. If you know <code>C</code>, <code>C++</code>, or <code>rust</code> you are good to go and if not that is what you will have to learn. I love <code>C</code>, it is a great language but learn <a href="https://www.rust-lang.org/"><code>rust</code></a> if you don't wanna accidentally blow up your system.</p>
<h1 id="hmm%2C-i-will-learn-those-later%2C-what-exactly-is-happening-here%3F" tabindex="-1">Hmm, I will learn those later, what exactly is happening here? <a class="direct-link" href="https://blog.meain.io/2017/ehh-webassembly/#hmm%2C-i-will-learn-those-later%2C-what-exactly-is-happening-here%3F">#</a></h1>
<p>Well, I am happy to break it down to you. Btw check out <a href="https://mbebenita.github.io/WasmExplorer/">Webassembly Playground</a>.</p>
<p>OK, let us see what happens here. Let us write a simple C function.</p>
<pre class="language-c"><code class="language-c"><span class="token keyword">int</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token keyword">int</span> x<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> x<span class="token operator">/</span><span class="token number">2</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p><em>I swear I know to write complex ones</em></p>
<p>So, yeah, with that out of the way let us see how the compiled version of this code looks like. It will look something like this.</p>
<pre class="language-nasm"><code class="language-nasm">wasm<span class="token operator">-</span>function<span class="token operator">[</span><span class="token number">0</span><span class="token operator">]</span>:<br /> sub <span class="token register variable">rsp</span>, <span class="token number">8</span> <span class="token comment">; 0x000000 48 83 ec 08</span><br /> mov <span class="token register variable">eax</span>, <span class="token register variable">edi</span> <span class="token comment">; 0x000004 8b c7</span><br /> shr <span class="token register variable">eax</span>, <span class="token number">0x1f</span> <span class="token comment">; 0x000006 c1 e8 1f</span><br /> add <span class="token register variable">eax</span>, <span class="token register variable">edi</span> <span class="token comment">; 0x000009 03 c7</span><br /> sar <span class="token register variable">eax</span>, <span class="token number">1</span> <span class="token comment">; 0x00000b d1 f8</span><br /> nop <span class="token comment">; 0x00000d 66 90</span><br /> add <span class="token register variable">rsp</span>, <span class="token number">8</span> <span class="token comment">; 0x00000f 48 83 c4 08</span><br /> ret <span class="token comment">; 0x000013 c3</span></code></pre>
<p>I kinda know assembly(had to learn it in college) and I can make kinda make sense of this but for a relief this is not the code that you will be debugging in the browser. What you will be debugging in the browser will be something known as <code>WAST</code>(Webassembly Syntax Tree) and that will look something like this.</p>
<pre><code>(module
(table 0 anyfunc)
(memory $0 1)
(export "memory" (memory $0))
(export "_Z3fooi" (func $_Z3fooi))
(func $_Z3fooi (param $0 i32) (result i32)
(i32.div_s
(get_local $0)
(i32.const 2)
)
)
)
</code></pre>
<p>Well, this is what it look like but this won't be what it actually is. Actually it will be a bit more different as it is actually implemented as a stack machine(something that can work by pushing or popping from the stack or doing an operation). That will look more like this.</p>
<pre><code>(module
(type $type0 (func (param i32) (result i32)))
(table 0 anyfunc)
(memory 1)
(export "memory" memory)
(export "_Z3fooi" $func0)
(func $func0 (param $var0 i32) (result i32)
get_local $var0
i32.const 2
i32.div_s
)
)
</code></pre>
<h1 id="hmm-cool%2C-but-i-can't-understand-wast-either" tabindex="-1">Hmm cool, but I can't understand <code>WAST</code> either <a class="direct-link" href="https://blog.meain.io/2017/ehh-webassembly/#hmm-cool%2C-but-i-can't-understand-wast-either">#</a></h1>
<p>Oh, wait. It is easy. Let me help you here(with my minimal knowledge).</p>
<p>The part we need to concentrate here is</p>
<pre><code>(func $func0 (param $var0 i32) (result i32)
get_local $var0
i32.const 2
i32.div_s
)
</code></pre>
<p>Most of the other lines are kinda like boilerplate code to initialize memory and stuff(yeah, stuff. I don't really know). Now if we see here, in the first line we can see that it is function called <code>$func0</code> which takes in an parameter <code>var0</code> of type int32 and outputs result of type int32. Makes sense?</p>
<p>OK, now with that out of the way, let us get to the other lines. As I mentioned earlier, <code>WAST</code> is implemented as a stack machine. So view the code with that in mind.
Initially we get the variable <code>var0</code> which was passed as argument to the function. Now we create an int32 constant of value 2. Now we divide the initial variable <code>var0</code> with the constant value 2. Well that is your <strong>x/2</strong>.</p>
<blockquote>
<p>You probably didn't need the explanation, but yeah just in case.</p>
</blockquote>
<h1 id="well%2C-i-guess-i-understand.-but-why-though%3F" tabindex="-1">Well, I guess I understand. But why though? <a class="direct-link" href="https://blog.meain.io/2017/ehh-webassembly/#well%2C-i-guess-i-understand.-but-why-though%3F">#</a></h1>
<p>Good question. As you know languages such as <code>C</code>, <code>C++</code> and <code>rust</code> are statically typed languages. This means you can leverage the static type information that is available int the code to optimize it. Fox example if we where to set the variable <code>x</code> in our initial program we could do a binary right shift instead of a div to divide by 2 and that is a much more efficient way to do it.</p>
<p>Let us see some code:</p>
<p><code>new C++ code:</code></p>
<pre class="language-c"><code class="language-c"><span class="token keyword">unsigned</span> <span class="token keyword">int</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token keyword">unsigned</span> <span class="token keyword">int</span> x<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> x<span class="token operator">/</span><span class="token number">2</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p><code>WAST output:</code></p>
<pre><code>(module
(type $type0 (func (param i32) (result i32)))
(table 0 anyfunc)
(memory 1)
(export "memory" memory)
(export "_Z3fooj" $func0)
(func $func0 (param $var0 i32) (result i32)
get_local $var0
i32.const 1
i32.shr_u
)
)
</code></pre>
<p>If you look closely in the last line in <code>WAST</code> output <code>i32.div_s</code> was changed to <code>i32.shr_u</code> which is a faster instruction. Well, in here this might not matter so much but when you are creating something bigger(game engines maybe?) it does matter.</p>
<h1 id="oh-snap%2C-so-is-js-gonna-die%3F" tabindex="-1">Oh snap, so is JS gonna die? <a class="direct-link" href="https://blog.meain.io/2017/ehh-webassembly/#oh-snap%2C-so-is-js-gonna-die%3F">#</a></h1>
<p>Nah, not really. The fact is you still need JS to handle the DOM. All you can do from <code>Webassembly</code> is to do what you could have done natively by uinsg JS api's for webassembly to call into <code>WASM</code> modules and get some result back.</p>
<h4 id="wait%2C-ain't-that-what-flash-did%3F" tabindex="-1">Wait, ain't that what flash did? <a class="direct-link" href="https://blog.meain.io/2017/ehh-webassembly/#wait%2C-ain't-that-what-flash-did%3F">#</a></h4>
<p>Well, it is but not really. In the case of flash it was run as a completely seperate module independent from the browser. But in the case of <code>WASM</code> you run that in a browser in a sandboxed environment. The issue with flash was that if you wanted to do something crazy on the system that the browser did not let you do, you could just call into flash and do it. But in the case of <code>WASM</code> you do not have the issue as it also runs inside the browser sndbox.</p>
<h1 id="woosh.-btw-is-there-a-better-guide-avaiable-other-than-this-piece-of-crap%3F" tabindex="-1">Woosh. Btw is there a better guide avaiable other than this piece of crap? <a class="direct-link" href="https://blog.meain.io/2017/ehh-webassembly/#woosh.-btw-is-there-a-better-guide-avaiable-other-than-this-piece-of-crap%3F">#</a></h1>
<p>Oh, definitely. Check out the blog <a href="https://hacks.mozilla.org/2017/02/a-cartoon-intro-to-webassembly/">A cartoon intro to WebAssembly</a> or the <a href="http://webassembly.org/">official page</a>. For more info on working with <code>WASM</code> and benchmarks check out the blog <a href="https://hackernoon.com/screamin-speed-with-webassembly-b30fac90cd92">Screamin’ Speed with WebAssembly</a></p>
Creating a tic-tac-toe game in Clojurescript using Reagent2017-09-28T00:00:00Zhttps://blog.meain.io/2017/tictactoe-clojurescript-reagent/<p>Whoosh, I am tired of JS all day long, let me try out something else. And yeah, I ended up here.
Actually I was kinda playing around with <code>Haskell</code> before I reached here. It feels pretty good to have a change from the usual stuff.
To be frank I think everyone should try out functional programming at some point of time.
Well, enough bullshit let us get to building it.</p>
<p><img src="https://blog.meain.io/img/tictactoe.png" alt="screenshot" /></p>
<blockquote>
<p>Fully working code: <a href="https://github.com/meain/tictactoe-clojurescript-reagent">tictactoe-clojurescript-reagent</a></p>
</blockquote>
<p>To start with <code>Clojrescript</code> is a <a href="https://wiki.haskell.org/Functional_programming">functional programming language</a>.
It derives from <a href="https://clojure.org/"><code>Clojure</code></a>.</p>
<blockquote>
<p>Random fact: <code>Clojure</code> runs on top of Java.</p>
</blockquote>
<h1 id="installation-and-setup" tabindex="-1">Installation and setup <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#installation-and-setup">#</a></h1>
<p>I want to add the setup instructions, but it is different for different systems and I am really lazy right now so go check out the webpage.</p>
<p>Just some pointers, you will need <code>Java</code> first. Then install <code>Clojurescript</code> and then <code>leiningen</code>.</p>
<p><a href="https://leiningen.org/"><code>Leiningen</code></a> is kinda like gulp.</p>
<p>Btw in mac it is:</p>
<pre class="language-shell"><code class="language-shell">brew <span class="token function">install</span> clojurescript<br />brew <span class="token function">install</span> leiningen</code></pre>
<h1 id="getting-started" tabindex="-1">Getting started <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#getting-started">#</a></h1>
<h2 id="setting-up-template" tabindex="-1">Setting up template <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#setting-up-template">#</a></h2>
<p>First you blah blah....</p>
<p>Run this command:</p>
<pre class="language-shell"><code class="language-shell">lein new figwheel ttt -- <span class="token parameter variable">--reagent</span></code></pre>
<p>It is kinda like <code>npm init</code> command but with <code>livereload</code> and <code>react</code> installed.</p>
<p>OK, what that does is create a new project with <a href="https://github.com/bhauman/lein-figwheel"><code>figwheel</code></a> and using <a href="https://github.com/reagent-project/reagent"><code>reagent</code></a></p>
<p><code>Figwheel</code> is like <code>gulp-livereload</code> plugin. <code>Reagent</code> is the <code>Clojurescript</code> wrapper arround Facebook's <code>React</code> JS framework.</p>
<p>Now you start the <code>lien</code> server ( which has hot reload ) by using the command:</p>
<pre class="language-shell"><code class="language-shell">lein figwheel</code></pre>
<p>You will get a file structure like this</p>
<pre><code>ttt
├── README.md
├── dev
│ └── user.clj
├── project.clj
├── resources
│ └── public
│ ├── css
│ │ └── style.css
│ └── index.html
└── src
└── ttt
└── core.cljs
6 directories, 6 files
</code></pre>
<p>The main file you will have to work on here is <code>src/ttt/core.cljs</code>. It is the file that we will add all the core logic into.</p>
<p><code>project.clj</code> is a configuration file, kinda like <code>package.json</code>.</p>
<h2 id="well%2C-let-us-see-some-basics-of-clojrescript" tabindex="-1">Well, let us see some basics of <code>Clojrescript</code> <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#well%2C-let-us-see-some-basics-of-clojrescript">#</a></h2>
<p>In the <code>core.cljs</code> file you will have:</p>
<pre class="language-clojure"><code class="language-clojure"><span class="token punctuation">(</span><span class="token keyword">ns</span> ttt.core<br /> <span class="token punctuation">(</span><span class="token symbol">:require</span> <span class="token punctuation">[</span>reagent.core <span class="token symbol">:as</span> reagent <span class="token symbol">:refer</span> <span class="token punctuation">[</span>atom<span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><br /><span class="token punctuation">(</span><span class="token function">enable-console-print!</span><span class="token punctuation">)</span><br /><br /><span class="token punctuation">(</span><span class="token keyword">println</span> <span class="token string">"This text is printed from src/ttt/core.cljs. Go ahead and edit it and see reloading in action."</span><span class="token punctuation">)</span><br /><br /><span class="token comment">;; define your app data so that it doesn't get over-written on reload</span><br /><br /><span class="token punctuation">(</span><span class="token keyword">defonce</span> app-state <span class="token punctuation">(</span><span class="token function">atom</span> <span class="token punctuation">{</span><span class="token symbol">:text</span> <span class="token string">"Hello world!"</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><br /><br /><span class="token punctuation">(</span><span class="token keyword">defn</span> hello-world <span class="token punctuation">[</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:div</span><br /> <span class="token punctuation">[</span><span class="token symbol">:h1</span> <span class="token punctuation">(</span><span class="token symbol">:text</span> <span class="token operator">@</span>app-state<span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:h3</span> <span class="token string">"Edit this and watch it change!"</span><span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">)</span><br /><br /><span class="token punctuation">(</span><span class="token function">reagent/render-component</span> <span class="token punctuation">[</span>hello-world<span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token keyword">.</span> js/document <span class="token punctuation">(</span><span class="token function">getElementById</span> <span class="token string">"app"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><br /><span class="token punctuation">(</span><span class="token keyword">defn</span> on-js-reload <span class="token punctuation">[</span><span class="token punctuation">]</span><br /> <span class="token comment">;; optionally touch your app-state to force rerendering depending on</span><br /> <span class="token comment">;; your application</span><br /> <span class="token comment">;; (swap! app-state update-in [:__figwheel_counter] inc)</span><br /><span class="token punctuation">)</span></code></pre>
<p>Let us see what all of these lines do.</p>
<h3 id="(ns-....)" tabindex="-1">(ns ....) <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#(ns-....)">#</a></h3>
<p>The top two lines is kinda your import statements. Mostly importing <code>Reagent</code> here.</p>
<h3 id="console.log()" tabindex="-1">console.log() <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#console.log()">#</a></h3>
<p>Well we have <code>println</code> instead of <code>console.log</code>. But since this is a functional language we use it like:</p>
<pre class="language-clojure"><code class="language-clojure"><span class="token punctuation">(</span><span class="token keyword">println</span> <span class="token string">"Stuff you wanna print"</span><span class="token punctuation">)</span></code></pre>
<p>This will get printed in the JS console in your browser.</p>
<h3 id="commenting" tabindex="-1">Commenting <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#commenting">#</a></h3>
<p>You can use <code>;</code> to start a comment.
Anything after this, used on start of a line or anywhere in the line is considered as a comment.</p>
<p>Example:</p>
<pre class="language-clojure"><code class="language-clojure"><span class="token comment">; I heard you like comments</span></code></pre>
<h3 id="app-state" tabindex="-1"><code>app-state</code> <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#app-state">#</a></h3>
<p>In line 10, what you see is a variable ( <a href="http://clojuredocs.org/clojure.core/atom"><code>atom</code></a> ) declaration. The difference between this variable and others is that it is an immutable variable which means you cannot modify it in place.</p>
<h3 id="(reagent%2Frender-component)" tabindex="-1">(reagent/render-component) <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#(reagent%2Frender-component)">#</a></h3>
<p><em>Renders component, duh.</em>
It renders the <code>hello-world</code> component on to the div with id <code>app</code>.</p>
<h3 id="(defn-hello-world-%5B%5D-)" tabindex="-1">(defn hello-world [] ) <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#(defn-hello-world-%5B%5D-)">#</a></h3>
<p>Well <code>hello-world</code> is a function that is return a html string kinda thing or more like return a JSX object ( if that is a thing ).
In <code>Clojurescript</code> world it is called <a href="https://github.com/weavejester/hiccup"><code>hiccup</code></a> like syntax. It has pretty much redid the html in a <code>Clojure</code> ish syntax.</p>
<p>Actually from <a href="http://reagent-project.github.io/"><code>Reagent</code></a>.</p>
<p>The defenition here:</p>
<pre class="language-clojure"><code class="language-clojure"><span class="token punctuation">[</span><span class="token symbol">:div</span><br /> <span class="token punctuation">[</span><span class="token symbol">:h1</span> <span class="token punctuation">(</span><span class="token symbol">:text</span> <span class="token operator">@</span>app-state<span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:h3</span> <span class="token string">"Edit this and watch it change!"</span><span class="token punctuation">]</span><span class="token punctuation">]</span></code></pre>
<p>gives you something like:</p>
<pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>div</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>h1</span><span class="token punctuation">></span></span>${app-state.text}<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>h1</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>h3</span><span class="token punctuation">></span></span>Edit this and watch it change!<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>h3</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>div</span><span class="token punctuation">></span></span></code></pre>
<h4 id="(%3Atext-%40app-state)" tabindex="-1">(:text @app-state) <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#(%3Atext-%40app-state)">#</a></h4>
<p>This is how you get a value from the immutable variable.</p>
<blockquote>
<p>You use <code>@</code> when using an atom</p>
</blockquote>
<p>Now if you have some value multiple levels deep you can use <a href="http://clojuredocs.org/clojure.core/get-in"><code>get-in</code></a></p>
<p>For example:</p>
<pre class="language-clojure"><code class="language-clojure">user=> <span class="token punctuation">(</span><span class="token keyword">def</span> m <span class="token punctuation">{</span><span class="token symbol">:username</span> <span class="token string">"sally"</span><br /> <span class="token symbol">:profile</span> <span class="token punctuation">{</span><span class="token symbol">:name</span> <span class="token string">"Sally Clojurian"</span><br /> <span class="token symbol">:address</span> <span class="token punctuation">{</span><span class="token symbol">:city</span> <span class="token string">"Austin"</span> <span class="token symbol">:state</span> <span class="token string">"TX"</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token punctuation">)</span><br /><span class="token operator">#</span>'user/m<br /><br />user=> <span class="token punctuation">(</span><span class="token function">get-in</span> m <span class="token punctuation">[</span><span class="token symbol">:profile</span> <span class="token symbol">:name</span><span class="token punctuation">]</span><span class="token punctuation">)</span><br /><span class="token string">"Sally Clojurian"</span><br />user=> <span class="token punctuation">(</span><span class="token function">get-in</span> m <span class="token punctuation">[</span><span class="token symbol">:profile</span> <span class="token symbol">:address</span> <span class="token symbol">:city</span><span class="token punctuation">]</span><span class="token punctuation">)</span><br /><span class="token string">"Austin"</span><br />user=> <span class="token punctuation">(</span><span class="token function">get-in</span> m <span class="token punctuation">[</span><span class="token symbol">:profile</span> <span class="token symbol">:address</span> <span class="token symbol">:zip-code</span><span class="token punctuation">]</span><span class="token punctuation">)</span><br /><span class="token boolean">nil</span><br />user=> <span class="token punctuation">(</span><span class="token function">get-in</span> m <span class="token punctuation">[</span><span class="token symbol">:profile</span> <span class="token symbol">:address</span> <span class="token symbol">:zip-code</span><span class="token punctuation">]</span> <span class="token string">"no zip code!"</span><span class="token punctuation">)</span><br /><span class="token string">"no zip code!"</span></code></pre>
<blockquote>
<p>Check out <a href="http://cljs.info/cheatsheet/">ClojureScript Cheatsheet</a> in case you get stuck.</p>
</blockquote>
<h1 id="building-the-game" tabindex="-1">Building the game <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#building-the-game">#</a></h1>
<p>Cool, now with all that basics out of the way let us get to building the actual game.</p>
<blockquote>
<p>I will be using the code from here (<a href="https://github.com/meain/tictactoe-clojurescript-reagent">tictactoe-clojurescript-reagent</a>)</p>
</blockquote>
<p>So as I said the main file you will be checking out will be <code>src/ttt/core.cljs</code>.
We will go line by line ( mostly ) from the above cited project's <code>core.cljs</code> file.</p>
<h2 id="lines-9---10" tabindex="-1">Lines 9 - 10 <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#lines-9---10">#</a></h2>
<pre class="language-clojure"><code class="language-clojure"><span class="token punctuation">(</span><span class="token keyword">defn</span> make-board <span class="token string">"Creates a new board. n denotes the size."</span> <span class="token punctuation">[</span>n<span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token function">vec</span> <span class="token punctuation">(</span><span class="token keyword">repeat</span> n <span class="token punctuation">(</span><span class="token function">vec</span> <span class="token punctuation">(</span><span class="token keyword">repeat</span> n <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p>In here we define a function which will, on call return a [n x n] matrix.
<code>(repeat n 0)</code> creates a list of n 0's. We turn that into a vector. Now we get one n dimensional array.
We create multiple copies of this to create an [n x n] matrix.</p>
<h2 id="lines-12---17" tabindex="-1">Lines 12 - 17 <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#lines-12---17">#</a></h2>
<pre class="language-clojure"><code class="language-clojure"><span class="token punctuation">(</span><span class="token keyword">def</span> board-size <span class="token number">3</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token keyword">defonce</span> app-state<br /> <span class="token punctuation">(</span><span class="token function">atom</span> <span class="token punctuation">{</span><span class="token symbol">:text</span> <span class="token string">":game"</span><br /> <span class="token symbol">:board</span> <span class="token punctuation">(</span><span class="token function">make-board</span> board-size<span class="token punctuation">)</span><br /> <span class="token comment">;; none win lose draw</span><br /> <span class="token symbol">:win</span> <span class="token string">"none"</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p>In here we are mainly defining variables. In line 12 we define the variable <code>board-size</code>.
In lines 13 to 17 we are creating an atom ( an immutable variable ) which contains a text, the current board and win state.</p>
<h2 id="lines-19---30" tabindex="-1">Lines 19 - 30 <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#lines-19---30">#</a></h2>
<pre class="language-clojure"><code class="language-clojure"><span class="token punctuation">(</span><span class="token keyword">defn</span> check-win <span class="token string">"Check for win and lose conditions"</span> <span class="token punctuation">[</span>user computer<span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">or</span> <span class="token punctuation">(</span><span class="token keyword">some</span> <span class="token operator">#</span><span class="token punctuation">(</span><span class="token keyword">=</span> board-size %<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>freq <span class="token punctuation">(</span><span class="token function">frequencies</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>el user<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">first</span> el<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">second</span> freq<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">some</span> <span class="token operator">#</span><span class="token punctuation">(</span><span class="token keyword">=</span> board-size %<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>freq <span class="token punctuation">(</span><span class="token function">frequencies</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>el user<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">second</span> el<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">second</span> freq<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">=</span> board-size <span class="token punctuation">(</span><span class="token function">get-in</span> <span class="token punctuation">(</span><span class="token function">frequencies</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>el user<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token keyword">first</span> el<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">second</span> el<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">[</span><span class="token boolean">true</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">=</span> board-size <span class="token punctuation">(</span><span class="token function">get-in</span> <span class="token punctuation">(</span><span class="token function">frequencies</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>el user<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token keyword">first</span> el<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">-</span> <span class="token punctuation">(</span><span class="token keyword">-</span> board-size <span class="token punctuation">(</span><span class="token keyword">second</span> el<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">[</span><span class="token boolean">true</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token function">swap!</span> app-state assoc <span class="token symbol">:win</span> <span class="token string">"win"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">or</span> <span class="token punctuation">(</span><span class="token keyword">some</span> <span class="token operator">#</span><span class="token punctuation">(</span><span class="token keyword">=</span> board-size %<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>freq <span class="token punctuation">(</span><span class="token function">frequencies</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>el computer<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">first</span> el<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">second</span> freq<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">some</span> <span class="token operator">#</span><span class="token punctuation">(</span><span class="token keyword">=</span> board-size %<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>freq <span class="token punctuation">(</span><span class="token function">frequencies</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>el computer<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">second</span> el<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">second</span> freq<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">=</span> board-size <span class="token punctuation">(</span><span class="token function">get-in</span> <span class="token punctuation">(</span><span class="token function">frequencies</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>el computer<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token keyword">first</span> el<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">second</span> el<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">[</span><span class="token boolean">true</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">=</span> board-size <span class="token punctuation">(</span><span class="token function">get-in</span> <span class="token punctuation">(</span><span class="token function">frequencies</span> <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>el computer<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token keyword">first</span> el<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">-</span> <span class="token punctuation">(</span><span class="token keyword">-</span> board-size <span class="token punctuation">(</span><span class="token keyword">second</span> el<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">[</span><span class="token boolean">true</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token function">swap!</span> app-state assoc <span class="token symbol">:win</span> <span class="token string">"lose"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">)</span></code></pre>
<p>Here we define the checks to see if the player or bot has won. Draw conditions are checked in another function.
The variables <code>user</code> and <code>computer</code> are values that are passed from the function <code>check-state</code> defined at line 32.
The contents of these variables are list of [i j] values for which the respective players have marked.</p>
<p>Once we have found out if there is a win or lose condition we use <code>swap!</code> and <code>assoc</code> to change the value of win inside of <code>app-state</code>.</p>
<h2 id="lines-32---49" tabindex="-1">Lines 32 - 49 <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#lines-32---49">#</a></h2>
<pre class="language-clojure"><code class="language-clojure"><span class="token punctuation">(</span><span class="token keyword">defn</span> check-state <span class="token punctuation">[</span><span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">[</span>board <span class="token punctuation">(</span><span class="token symbol">:board</span> <span class="token operator">@</span>app-state<span class="token punctuation">)</span><br /> remaining <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>i <span class="token punctuation">(</span><span class="token keyword">range</span> board-size<span class="token punctuation">)</span><br /> j <span class="token punctuation">(</span><span class="token keyword">range</span> board-size<span class="token punctuation">)</span><br /> <span class="token symbol">:when</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token function">get-in</span> board <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><span class="token punctuation">)</span><br /> user <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>i <span class="token punctuation">(</span><span class="token keyword">range</span> board-size<span class="token punctuation">)</span><br /> j <span class="token punctuation">(</span><span class="token keyword">range</span> board-size<span class="token punctuation">)</span><br /> <span class="token symbol">:when</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token function">get-in</span> board <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><span class="token punctuation">)</span><br /> computer <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>i <span class="token punctuation">(</span><span class="token keyword">range</span> board-size<span class="token punctuation">)</span><br /> j <span class="token punctuation">(</span><span class="token keyword">range</span> board-size<span class="token punctuation">)</span><br /> <span class="token symbol">:when</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token function">get-in</span> board <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token keyword">count</span> remaining<span class="token punctuation">)</span> <span class="token number">0</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token function">swap!</span> app-state assoc <span class="token symbol">:win</span> <span class="token string">"draw"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token function">check-win</span> user computer<span class="token punctuation">)</span><br /> <span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p>This function <code>check-state</code> along with the above one is used to determine the win, lose, draw or none setting of the <code>:win</code> setting.
This variable <code>:win</code> is what is then used in order to set the message on the screen.</p>
<p>What we use in our matrix to denote played position is 1 and 2 for user and computer respectively.
In this function we get the positions which are remaining ie not marked, positions played by the user and positions played by the computer into the variables <code>remaining</code>, <code>user</code> and <code>computer</code>.</p>
<p>If no elements are present in the <code>remaining</code> variable we set the condition as draw. Win lose conditions are checked in the <code>check-win</code> function which is called later on.</p>
<h2 id="lines-51---63" tabindex="-1">Lines 51 - 63 <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#lines-51---63">#</a></h2>
<pre class="language-clojure"><code class="language-clojure"><span class="token punctuation">(</span><span class="token keyword">defn</span> computer-move <span class="token punctuation">[</span><span class="token punctuation">]</span><br /> <span class="token comment">;; choose a random unplayed block</span><br /> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">[</span>board <span class="token punctuation">(</span><span class="token symbol">:board</span> <span class="token operator">@</span>app-state<span class="token punctuation">)</span><br /> remaining <span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>i <span class="token punctuation">(</span><span class="token keyword">range</span> board-size<span class="token punctuation">)</span><br /> j <span class="token punctuation">(</span><span class="token keyword">range</span> board-size<span class="token punctuation">)</span><br /> <span class="token symbol">:when</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token function">get-in</span> board <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><span class="token punctuation">)</span><br /> move <span class="token punctuation">(</span><span class="token function">rand-nth</span> remaining<span class="token punctuation">)</span><br /> path <span class="token punctuation">(</span><span class="token keyword">into</span> <span class="token punctuation">[</span><span class="token symbol">:board</span><span class="token punctuation">]</span> move<span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token function">swap!</span> app-state assoc-in path <span class="token number">2</span><span class="token punctuation">)</span><br /> <span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token function">check-state</span><span class="token punctuation">)</span><br /> <span class="token punctuation">)</span></code></pre>
<p>This function is used to compute a move for the computer to play.
What it does it it picks out the remaining positions in the game-matrix and store it to <code>remaining</code> variable.
Then we chose a random value from the <code>remaining</code> variable and make the change to the board.</p>
<h2 id="lines-65---121" tabindex="-1">Lines 65 - 121 <a class="direct-link" href="https://blog.meain.io/2017/tictactoe-clojurescript-reagent/#lines-65---121">#</a></h2>
<pre class="language-clojure"><code class="language-clojure"><span class="token punctuation">(</span><span class="token keyword">defn</span> block <span class="token punctuation">[</span>color i j<span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:div</span> <span class="token punctuation">{</span><span class="token symbol">:style</span> <span class="token punctuation">{</span><span class="token symbol">:background-color</span> color<br /> <span class="token symbol">:width</span> <span class="token string">"100px"</span><br /> <span class="token symbol">:height</span> <span class="token string">"100px"</span><br /> <span class="token symbol">:border</span> <span class="token string">"5px solid #fff"</span><span class="token punctuation">}</span><br /> <span class="token symbol">:on-click</span> <span class="token punctuation">(</span><span class="token keyword">fn</span> <span class="token punctuation">[</span>e<span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">and</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token number">0</span> <span class="token punctuation">(</span><span class="token function">get-in</span> <span class="token operator">@</span>app-state <span class="token punctuation">[</span><span class="token symbol">:board</span> i j<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token symbol">:win</span> <span class="token operator">@</span>app-state<span class="token punctuation">)</span> <span class="token string">"none"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token function">swap!</span> app-state assoc-in <span class="token punctuation">[</span><span class="token symbol">:board</span> i j<span class="token punctuation">]</span> <span class="token number">1</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token function">check-state</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">=</span> <span class="token punctuation">(</span><span class="token symbol">:win</span> <span class="token operator">@</span>app-state<span class="token punctuation">)</span> <span class="token string">"none"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token function">computer-move</span><span class="token punctuation">)</span><br /> <span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token punctuation">]</span><span class="token punctuation">)</span><br /><br /><span class="token punctuation">(</span><span class="token keyword">defn</span> blank <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token function">block</span> <span class="token string">"#f5f5f5"</span> i j<span class="token punctuation">)</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token keyword">defn</span> cross <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token function">block</span> <span class="token string">"#FF7043"</span> i j<span class="token punctuation">)</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token keyword">defn</span> circle <span class="token punctuation">[</span>i j<span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token function">block</span> <span class="token string">"#FFEE58"</span> i j<span class="token punctuation">)</span><span class="token punctuation">)</span><br /><br /><span class="token punctuation">(</span><span class="token keyword">defn</span> render-board <span class="token punctuation">[</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:div</span> <span class="token punctuation">{</span><span class="token symbol">:style</span> <span class="token punctuation">{</span><span class="token symbol">:display</span> <span class="token string">"flex"</span> <span class="token symbol">:flex-wrap</span> <span class="token string">"wrap"</span><span class="token punctuation">}</span><span class="token punctuation">}</span><br /> <span class="token punctuation">(</span>doall<span class="token punctuation">(</span><span class="token keyword">for</span> <span class="token punctuation">[</span>i <span class="token punctuation">(</span><span class="token keyword">range</span> board-size<span class="token punctuation">)</span><br /> j <span class="token punctuation">(</span><span class="token keyword">range</span> board-size<span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token function">case</span> <span class="token punctuation">(</span><span class="token function">get-in</span> <span class="token operator">@</span>app-state <span class="token punctuation">[</span><span class="token symbol">:board</span> i j<span class="token punctuation">]</span><span class="token punctuation">)</span><br /> <span class="token number">0</span> <span class="token operator">^</span><span class="token punctuation">{</span><span class="token symbol">:key</span> <span class="token punctuation">(</span><span class="token keyword">str</span> i <span class="token string">"-"</span> j<span class="token punctuation">)</span><span class="token punctuation">}</span> <span class="token punctuation">[</span>blank i j<span class="token punctuation">]</span><br /> <span class="token number">1</span> <span class="token operator">^</span><span class="token punctuation">{</span><span class="token symbol">:key</span> <span class="token punctuation">(</span><span class="token keyword">str</span> i <span class="token string">"-"</span> j<span class="token punctuation">)</span><span class="token punctuation">}</span> <span class="token punctuation">[</span>cross i j<span class="token punctuation">]</span><br /> <span class="token number">2</span> <span class="token operator">^</span><span class="token punctuation">{</span><span class="token symbol">:key</span> <span class="token punctuation">(</span><span class="token keyword">str</span> i <span class="token string">"-"</span> j<span class="token punctuation">)</span><span class="token punctuation">}</span> <span class="token punctuation">[</span>circle i j<span class="token punctuation">]</span><br /> <span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">]</span><span class="token punctuation">)</span><br /><br /><span class="token punctuation">(</span><span class="token keyword">defn</span> app <span class="token punctuation">[</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:div</span> <span class="token punctuation">{</span><span class="token symbol">:style</span> <span class="token punctuation">{</span><span class="token symbol">:text-align</span> <span class="token string">"center"</span><span class="token punctuation">}</span><span class="token punctuation">}</span><br /> <span class="token punctuation">[</span><span class="token symbol">:div</span><br /> <span class="token punctuation">[</span><span class="token symbol">:h1</span> <span class="token punctuation">{</span><span class="token symbol">:style</span> <span class="token punctuation">{</span><span class="token symbol">:display</span> <span class="token string">"block"</span> <span class="token symbol">:float</span> <span class="token string">"left"</span><span class="token punctuation">}</span><span class="token punctuation">}</span> <span class="token punctuation">(</span><span class="token symbol">:text</span> <span class="token operator">@</span>app-state<span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:h1</span> <span class="token punctuation">{</span><span class="token symbol">:style</span><span class="token punctuation">{</span><span class="token symbol">:background-color</span> <span class="token string">"#f5f5f5"</span><br /> <span class="token symbol">:display</span> <span class="token string">"block"</span><br /> <span class="token symbol">:float</span> <span class="token string">"right"</span><span class="token punctuation">}</span><span class="token punctuation">}</span> <span class="token string">"tic-tac-toe"</span><span class="token punctuation">]</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:div.clearfix</span> <span class="token punctuation">{</span><span class="token symbol">:style</span> <span class="token punctuation">{</span><span class="token symbol">:clear</span> <span class="token string">"both"</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:h3</span> <span class="token punctuation">{</span><span class="token symbol">:style</span> <span class="token punctuation">{</span><span class="token symbol">:width</span> <span class="token string">"100%"</span> <span class="token symbol">:text-align</span> <span class="token string">"center"</span><span class="token punctuation">}</span><span class="token punctuation">}</span> <span class="token string">"Let us play some "</span><br /> <span class="token punctuation">[</span><span class="token symbol">:code</span> <span class="token punctuation">{</span><span class="token symbol">:style</span> <span class="token punctuation">{</span><span class="token symbol">:font-family</span> <span class="token string">"cursive"</span><span class="token punctuation">}</span><span class="token punctuation">}</span> <span class="token string">"tic-tac-toe"</span><span class="token punctuation">]</span> <span class="token string">" now"</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:center</span> <span class="token punctuation">[</span><span class="token symbol">:div</span> <span class="token punctuation">{</span><span class="token symbol">:style</span> <span class="token punctuation">{</span><span class="token symbol">:font-size</span> <span class="token string">"20px"</span><span class="token punctuation">}</span><span class="token punctuation">}</span> <span class="token punctuation">(</span><span class="token symbol">:win</span> <span class="token operator">@</span>app-state<span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:div.play-area</span> <span class="token punctuation">{</span><span class="token symbol">:style</span> <span class="token punctuation">{</span><span class="token symbol">:width</span> <span class="token punctuation">(</span><span class="token keyword">str</span> <span class="token punctuation">(</span><span class="token keyword">*</span> <span class="token number">110</span> board-size<span class="token punctuation">)</span> <span class="token string">"px"</span><span class="token punctuation">)</span><br /> <span class="token symbol">:height</span> <span class="token punctuation">(</span><span class="token keyword">str</span> <span class="token punctuation">(</span><span class="token keyword">*</span> <span class="token number">110</span> board-size<span class="token punctuation">)</span> <span class="token string">"px"</span><span class="token punctuation">)</span><br /> <span class="token symbol">:background-color</span> <span class="token string">"#ded"</span><br /> <span class="token symbol">:cursor</span> <span class="token string">"pointer"</span><br /> <span class="token symbol">:display</span> <span class="token string">"inline-block"</span><span class="token punctuation">}</span><span class="token punctuation">}</span><br /> <span class="token punctuation">[</span>render-board<span class="token punctuation">]</span><br /> <span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token symbol">:center</span><br /> <span class="token punctuation">[</span><span class="token symbol">:button</span> <span class="token punctuation">{</span><span class="token symbol">:on-click</span> <span class="token punctuation">(</span><span class="token keyword">fn</span> <span class="token punctuation">[</span>e<span class="token punctuation">]</span><br /> <span class="token punctuation">(</span><span class="token function">swap!</span> app-state assoc <span class="token symbol">:board</span> <span class="token punctuation">(</span><span class="token function">make-board</span> board-size<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token function">swap!</span> app-state assoc <span class="token symbol">:win</span> <span class="token string">"none"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">)</span><br /> <span class="token symbol">:style</span> <span class="token punctuation">{</span><span class="token symbol">:font-size</span> <span class="token string">"30px"</span><br /> <span class="token symbol">:font-family</span> <span class="token string">"monaco, monospace"</span><br /> <span class="token symbol">:margin-top</span> <span class="token string">"20px"</span><span class="token punctuation">}</span><span class="token punctuation">}</span> <span class="token string">"New Game"</span><span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">)</span></code></pre>
<p>This is the UI definition of the game.
The function <code>block</code> gives a hiccup like object of a single block.
The function <code>blank</code> <code>cross</code> and <code>circle</code> are just simple wrappers around <code>block</code> to give different color blocks.</p>
<blockquote>
<p>We use colors instead of symbols just because they are easy.</p>
</blockquote>
<p>The <code>render-board</code> function is used to render the whole board using the <code>board</code> value in <code>app-state</code> atom.
The definition is mostly the <code>hiccup</code> like syntax and I hope that makes sense to you.
Towards the end we also add a button to reset the game which has an <code>on-click</code> lister added to it which changes the value of board into a new one and reset the <code>win</code> value to none.</p>
<p>And that basically wraps up the whole code. :)</p>
Take a picture of you every time you open your laptop2018-01-09T00:00:00Zhttps://blog.meain.io/2018/profiling-yourself/<blockquote>
<p>Take a photo every time the lid is open using the laptop camera.</p>
</blockquote>
<p>Hey, ever seen those time-lapse videos of plants growing up in a BBC documentary and thought wish I could do that for me.</p>
<p>You haven't? I knew.</p>
<p>But here is how you would do it if you had thought of doing it. ( in OSX )</p>
<blockquote>
<p>It also helped me to see who tried to open up my laptop when I was not around.</p>
</blockquote>
<p>I started doing this in march of 2016 ( now have about 3700 images ) and from time to time try creating something like a time-lapse.
It is kinda fun. Maybe, at least for some.</p>
<p><img src="https://blog.meain.io/img/loginimages-screenshot.png" alt="screenshot" /></p>
<h1 id="how-to" tabindex="-1">How to <a class="direct-link" href="https://blog.meain.io/2018/profiling-yourself/#how-to">#</a></h1>
<h3 id="step-1" tabindex="-1">Step 1 <a class="direct-link" href="https://blog.meain.io/2018/profiling-yourself/#step-1">#</a></h3>
<p>Install <a href="https://github.com/rharder/imgnap">imagesnap</a>.</p>
<h3 id="step-2" tabindex="-1">Step 2 <a class="direct-link" href="https://blog.meain.io/2018/profiling-yourself/#step-2">#</a></h3>
<p>Keep this in a file named <code>imageme.sh</code> in a folder any folder.</p>
<pre class="language-shell"><code class="language-shell"><span class="token shebang important">#!/bin/sh</span><br /><span class="token assign-left variable">DATE</span><span class="token operator">=</span><span class="token variable"><span class="token variable">$(</span><span class="token function">date</span> <span class="token string">"+%Y-%m-%d_%H-%M-%S"</span><span class="token variable">)</span></span><br />imagesnap <span class="token parameter variable">-w</span> <span class="token number">2.00</span> <span class="token string">"<span class="token environment constant">$HOME</span>/.loginimages/<span class="token variable">$DATE</span>.jpg"</span></code></pre>
<p>What the above script does is it creates a date string with year-month-day-hour-minute-second and now take a picture using <code>imagesnap</code> and save it to your home directory.</p>
<blockquote>
<p>The <code>-w 2.00</code> is a wait time so as to make sure we get your photo.</p>
</blockquote>
<h3 id="step-3" tabindex="-1">Step 3 <a class="direct-link" href="https://blog.meain.io/2018/profiling-yourself/#step-3">#</a></h3>
<p>Install <a href="http://www.bernhard-baehr.de/">sleepwatcher</a>.</p>
<p>You could download the binary from the above webpage or install using <code>homebrew</code> (recommended).</p>
<p><code>brew install sleepwatcher</code></p>
<h3 id="step-4" tabindex="-1">Step 4 <a class="direct-link" href="https://blog.meain.io/2018/profiling-yourself/#step-4">#</a></h3>
<p>Now to actually run the script on wakeup.</p>
<p>Add this to file <code>/etc/rc.wakeup</code></p>
<pre class="language-shell"><code class="language-shell"><span class="token shebang important">#!/bin/bash</span><br /><span class="token comment"># Run the following script on wakeup</span><br /><span class="token assign-left variable"><span class="token environment constant">PATH</span></span><span class="token operator">=</span><span class="token environment constant">$PATH</span>:/usr/local/bin<br />/Users/<span class="token environment constant">$USER</span>/.imagescript/imageme.sh</code></pre>
<p>Here, replace <code>/Users/$USER/.imagescript/imageme.sh</code> with the path to where you saved the <code>imageme.sh</code> script.</p>
<h3 id="step-5" tabindex="-1">Step 5 <a class="direct-link" href="https://blog.meain.io/2018/profiling-yourself/#step-5">#</a></h3>
<p>Take a nap. zzzz.</p>
Connecting `Redux` to `React`, simplified2018-01-24T00:00:00Zhttps://blog.meain.io/2018/connect-redux-to-react/<p>So, you have been working on your <code>React</code> project.
Maybe you have heard about <code>Redux</code> and how it can make it all better and need some help getting started.</p>
<h1 id="what-is-redux%3F" tabindex="-1">What is <code>Redux</code>? <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#what-is-redux%3F">#</a></h1>
<p>Hmm, good question. It is like this master thingy which handles the state of all your components rather than handling the state in the components. So what you do is when you need to make a change that affects something else ( ie something you would store in the state or pass up to the parent component ) you pass it over to <code>Redux</code> ( loosely speaking ). From there you can user <code>Redux</code> to send the data to where it is needed.</p>
<!-- ![redux flow](/img/redux.png) -->
<div style="text-align:center"><img src="https://blog.meain.io/img/redux.png" /></div>
<h1 id="so%2C-how-do-you-connect%3F" tabindex="-1">So, how do you connect? <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#so%2C-how-do-you-connect%3F">#</a></h1>
<h2 id="create-react-project" tabindex="-1">Create <code>React</code> project <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#create-react-project">#</a></h2>
<p>Let us get stated by creating a <code>React</code> project. I am using <a href="https://github.com/facebookincubator/create-react-app">create-react-app</a>.</p>
<pre class="language-shell"><code class="language-shell">create-react-app redux-example</code></pre>
<p>This will bootstrap your react project, but without <code>Redux</code>. Your file structure will look something like:</p>
<pre><code>.
├── README.md
├── node_modules
├── package.json
├── public
├── src
└── yarn.lock
</code></pre>
<h2 id="install-redux-and-react-redux" tabindex="-1">Install <code>redux</code> and <code>react-redux</code> <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#install-redux-and-react-redux">#</a></h2>
<p>Now we install the <code>redux</code> library as well as <code>react-redux</code> library to connect react and redux together.</p>
<h4 id="redux" tabindex="-1"><code>redux</code> <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#redux">#</a></h4>
<pre class="language-shell"><code class="language-shell"><span class="token comment"># npm</span><br /><span class="token function">npm</span> <span class="token function">install</span> redux<br /><br /><span class="token comment"># yarn</span><br /><span class="token function">yarn</span> <span class="token function">add</span> redux</code></pre>
<h4 id="react-redux" tabindex="-1"><code>react-redux</code> <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#react-redux">#</a></h4>
<pre class="language-shell"><code class="language-shell"><span class="token comment"># npm</span><br /><span class="token function">npm</span> <span class="token function">install</span> react-redux<br /><br /><span class="token comment"># yarn</span><br /><span class="token function">yarn</span> <span class="token function">add</span> react-redux</code></pre>
<h2 id="creating-necessary-files" tabindex="-1">Creating necessary files <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#creating-necessary-files">#</a></h2>
<p>Create <code>redux.js</code> file in <code>/src</code> directory.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">import</span> <span class="token punctuation">{</span> createStore <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">'redux'</span><span class="token punctuation">;</span><br /><br /><span class="token comment">// create initial state</span><br /><span class="token keyword">const</span> initialState <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><br /><br /><span class="token comment">// create reducer</span><br /><span class="token keyword">const</span> <span class="token function-variable function">reducer</span> <span class="token operator">=</span> <span class="token punctuation">(</span> <span class="token parameter">state <span class="token operator">=</span> initialState<span class="token punctuation">,</span> action</span> <span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token comment">// branch using switch</span><br /> <span class="token keyword">switch</span><span class="token punctuation">(</span>action<span class="token punctuation">.</span>type<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">case</span> <span class="token string">'FIRST-ACTION'</span><span class="token operator">:</span><br /> state <span class="token operator">=</span> <span class="token punctuation">{</span><br /> <span class="token operator">...</span>state<span class="token punctuation">,</span><br /> <span class="token operator">...</span>action<span class="token punctuation">.</span>payload<br /> <span class="token punctuation">}</span><br /> <span class="token keyword">break</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">return</span> state<br /><span class="token punctuation">}</span><br /><br /><span class="token comment">// create store</span><br /><span class="token keyword">const</span> store <span class="token operator">=</span> <span class="token function">createStore</span><span class="token punctuation">(</span>reducer<span class="token punctuation">)</span><br /><span class="token comment">// export</span><br /><span class="token keyword">export</span> <span class="token keyword">default</span> store</code></pre>
<blockquote>
<p>You can use <a href="https://redux.js.org/docs/api/combineReducers.html">combine-reducers</a> combine multiple reducers</p>
</blockquote>
<h2 id="add-redux-into-the-react-mix" tabindex="-1">Add redux into the react mix <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#add-redux-into-the-react-mix">#</a></h2>
<p>In <code>/src/index.js</code></p>
<h4 id="add-import-for-react-redux-and-store" tabindex="-1">Add import for <code>react-redux</code> and <code>store</code> <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#add-import-for-react-redux-and-store">#</a></h4>
<pre class="language-js"><code class="language-js"><span class="token keyword">import</span> <span class="token punctuation">{</span> Provider <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">'react-redux'</span><br /><br /><span class="token keyword">import</span> store <span class="token keyword">from</span> <span class="token string">'./redux.js'</span></code></pre>
<h4 id="change-the-render-function-to" tabindex="-1">Change the render function to <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#change-the-render-function-to">#</a></h4>
<pre class="language-jsx"><code class="language-jsx">ReactDOM<span class="token punctuation">.</span><span class="token function">render</span><span class="token punctuation">(</span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span><span class="token class-name">Provider</span></span> <span class="token attr-name">store</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>store<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"><br /> </span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span><span class="token class-name">App</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"><br /> </span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span><span class="token class-name">Provider</span></span><span class="token punctuation">></span></span><span class="token punctuation">,</span><br /> document<span class="token punctuation">.</span><span class="token function">getElementById</span><span class="token punctuation">(</span><span class="token string">'root'</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<h2 id="use-'em-in-the-code" tabindex="-1">Use 'em in the code <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#use-'em-in-the-code">#</a></h2>
<p>In your component file</p>
<p>Import <code>connect</code> from <code>react-redux</code></p>
<pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> <span class="token punctuation">{</span> connect <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">'react-redux'</span></code></pre>
<p>Now at the bottom, do some <em>magic</em> to add stuff from redux as props.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> <span class="token function-variable function">mapStateToProps</span> <span class="token operator">=</span> <span class="token parameter">state</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">user</span><span class="token operator">:</span> state<span class="token punctuation">.</span>user<span class="token punctuation">,</span><br /> <span class="token literal-property property">threads</span><span class="token operator">:</span> state<span class="token punctuation">.</span>threads<br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">const</span> <span class="token function-variable function">mapDispatchToProps</span> <span class="token operator">=</span> <span class="token parameter">dispatch</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token punctuation">{</span><br /> <span class="token function-variable function">userChanged</span><span class="token operator">:</span> <span class="token parameter">threads</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token function">dispatch</span><span class="token punctuation">(</span><span class="token punctuation">{</span><br /> <span class="token literal-property property">type</span><span class="token operator">:</span> <span class="token string">'USER_UPDATED'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">payload</span><span class="token operator">:</span> user<br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token function">connect</span><span class="token punctuation">(</span>mapStateToProps<span class="token punctuation">,</span> mapDispatchToProps<span class="token punctuation">)</span><span class="token punctuation">(</span>App<span class="token punctuation">)</span></code></pre>
<h1 id="and-there-you-go-%3B)" tabindex="-1">And there you go ;) <a class="direct-link" href="https://blog.meain.io/2018/connect-redux-to-react/#and-there-you-go-%3B)">#</a></h1>
<p>I have tried to keep it really simple, more like a cheat sheet rather than an explanation because I think that is more important for most people.
Feel free to ping me if you need any help.</p>
Quickly go to project root2018-11-04T00:00:00Zhttps://blog.meain.io/2018/go-to-root-of-git-project/<p>Let us say you went deep in to a highly nested project structure and want to get back to the project root.</p>
<p>Sure you could go <code>cd ../<TAB></code>, nah not here <code>cd ../../<TAB></code>, not here either .....
But there is a much better way to do this.</p>
<p>You can leverage <code>git</code> to find where the project root is.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">git</span> rev-parse --show-toplevel <span class="token operator"><span class="token file-descriptor important">2</span>></span> /dev/null</code></pre>
<p>This gives you the project root location.
Now you can make this into a fuction and source it.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function-name function">root</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token builtin class-name">cd</span> <span class="token variable"><span class="token variable">$(</span><span class="token function">git</span> rev-parse --show-toplevel <span class="token operator"><span class="token file-descriptor important">2</span>></span> /dev/null<span class="token variable">)</span></span><br /><span class="token punctuation">}</span></code></pre>
<p>Now you can type <code>root</code> and go to your project root.</p>
<p><em>Sweet right?</em></p>
Docker basics2018-11-09T00:00:00Zhttps://blog.meain.io/2018/docker-basics-react-app/<p>Hi, the idea here is to introduce you to how docker works in very basic terms.</p>
<p>We will go through how to create a docker file and how to run it and stuff like that. But just the basics, just enough
to get you started on docker so that you can take on from there by yourself.</p>
<p>I am not going to discuss as to how to install docker. Just go and look it up on their website.</p>
<h1 id="what-is-a-dockerfile%3F" tabindex="-1">What is a <code>Dockerfile</code>? <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#what-is-a-dockerfile%3F">#</a></h1>
<p>It is a file which will let you configure how you need the project to be set up to be used.
This file specifies stuff like the operating system to the dependencies we need to run our project.</p>
<p>In order to understand what a docker file contains let us see a sample docker file</p>
<pre class="language-docker"><code class="language-docker"><span class="token instruction"><span class="token keyword">FROM</span> node:11.1.0</span><br /><br /><span class="token instruction"><span class="token keyword">COPY</span> . /code</span><br /><span class="token instruction"><span class="token keyword">WORKDIR</span> /code</span><br /><br /><span class="token instruction"><span class="token keyword">RUN</span> npm i</span><br /><span class="token instruction"><span class="token keyword">CMD</span> npm start</span></code></pre>
<p>This will be one of the most minimal docker files that you will see, but does introduce you to the basics concepts that
you need to build your own <code>Dockerfile</code>.</p>
<h1 id="dude%2C-what-the-heck-do-the-stuff-in-there-mean%3F" tabindex="-1">Dude, what the heck do the stuff in there mean? <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#dude%2C-what-the-heck-do-the-stuff-in-there-mean%3F">#</a></h1>
<p>Well, any <code>Dockerfile</code> is composed of multiple commands that you use to set up the environment for your project to run.</p>
<h2 id="from" tabindex="-1"><code>FROM</code> <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#from">#</a></h2>
<p>The initial command <code>FROM</code> is used to choose the base image. You could actually go with something like ubuntu and then
install node on top of it, but the node community provides a simple small image of node with any of the node version
that you need. It makes your life much easier if you are relying just on <code>node</code>.
The same applies for stuff like <code>python</code> or even for thing like <code>mysql</code> and stuff.</p>
<p>So, here you just choose the version <code>11.1.0</code> of node.</p>
<h2 id="copy" tabindex="-1"><code>COPY</code> <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#copy">#</a></h2>
<p>This command copies over a file or folder to a specific location inside the container.</p>
<p>There are other commands like <code>ADD</code> and <code>VOLUME</code>.</p>
<p><code>ADD</code> is essentially <code>COPY</code> but you can "copy" over a link in which case it download it. Also, you can "copy" over an
archive, in which case it extracts it. But if all you want is just a file or folder copy, just stick to <code>ADD</code>.</p>
<p>In the line <code>COPY . /code</code> we copy over all the content in the current directory to <code>/code</code> inside the container.</p>
<p>And <code>VOLUME</code> is use to mount directories. In the case of <code>VOLUME</code>, the data will be in sync between host and container.</p>
<h2 id="workdir" tabindex="-1"><code>WORKDIR</code> <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#workdir">#</a></h2>
<p>This command is used to switch over to a different directory. Just think about it as a simple <code>cd</code> but inside the docker
container.</p>
<p>Here we just switch over to the <code>/code</code> directory.</p>
<h2 id="run-and-cmd" tabindex="-1"><code>RUN</code> and <code>CMD</code> <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#run-and-cmd">#</a></h2>
<p>Well, the next two lines specifies commands for docker to run, and since we have set the working directory to be <code>/code</code>, we
run the commands in that directory.</p>
<p><strong>But why two commands.</strong></p>
<p>I'll explain. <code>RUN</code> command is executed when you build the docker container. Well, I guess now is a good time to tell you
that docker running thingy happens in two steps. But more on that later. You first build the container and in the next
step you run the container.</p>
<p><code>CMD</code> is only triggered when you run the container.</p>
<h2 id="there-are-more..." tabindex="-1">There are more... <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#there-are-more...">#</a></h2>
<p>There are more commands that you will use in a <code>Dockerfile</code>, but this should be good enough for now.</p>
<h1 id="hmm%2C-ok.-got-the-dockerfile-ready.-how-to-run-it%3F" tabindex="-1">Hmm, OK. Got the <code>Dockerfile</code> ready. How to run it? <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#hmm%2C-ok.-got-the-dockerfile-ready.-how-to-run-it%3F">#</a></h1>
<p>Well, as I said, docker thingy is composed of two parts. One to build the container, another one to run the
container.</p>
<h2 id="building-the-docker-container" tabindex="-1">Building the docker container <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#building-the-docker-container">#</a></h2>
<p>To do that you do</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">docker</span> build <span class="token builtin class-name">.</span></code></pre>
<p>Well, this will build your container and give you and image id. Buttt... it is better you give your image a name (<code>tag</code>
as it is called). For that you do</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">docker</span> build <span class="token parameter variable">-t</span> name <span class="token builtin class-name">.</span></code></pre>
<p>This will build your container with the <code>tag</code> of name.</p>
<h2 id="running-the-image-that-you-built" tabindex="-1">Running the image that you built <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#running-the-image-that-you-built">#</a></h2>
<p>To do this, you run</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">docker</span> run <span class="token parameter variable">-t</span> name</code></pre>
<p>This is why you had to give your image a <code>tag</code>. It comes in handy when you have to run it.</p>
<p>Well, now you have it running, but how do you access it?? Good question.</p>
<p>Well, you have to map the port inside the docker container to one outside.
Let us say that your node app starts at <code>8000</code>. You can map the <code>8000</code> inside the container to <code>3000</code> outside by doing</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">docker</span> run <span class="token parameter variable">-t</span> name <span class="token parameter variable">-p</span> <span class="token number">3000</span>:8000</code></pre>
<p>Now if you head over to <a href="http://localhost:3000/">localhost:3000</a> you can see your site live.</p>
<h1 id="extras" tabindex="-1">Extras <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#extras">#</a></h1>
<p>Well, this section is mainly why I wanna write this blog in the first place. But thought I would give an introduction
since I am gonna write this anyway.</p>
<h2 id=".dockerignore" tabindex="-1"><code>.dockerignore</code> <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#.dockerignore">#</a></h2>
<p>This is more or less like <code>.gitignore</code>. Well add all the files and folder that we want docker to ignore into this.</p>
<p>A sample <code>.dockerignore</code> file will look something like</p>
<pre><code>dist
node_modules
.cache
.git
</code></pre>
<p>Having something like this will make sure that docker does not care about these files.</p>
<h2 id="docker-caching" tabindex="-1">Docker caching <a class="direct-link" href="https://blog.meain.io/2018/docker-basics-react-app/#docker-caching">#</a></h2>
<p>Docker caches the container after each build step.</p>
<p>Let us see a trivial example where we can put this to good use.</p>
<p>Instead of writing a <code>Dockerfile</code> like above, we do something like this</p>
<pre class="language-docker"><code class="language-docker"><span class="token instruction"><span class="token keyword">FROM</span> node:11.1.0</span><br /><br /><span class="token instruction"><span class="token keyword">COPY</span> package.json /code/package.json</span><br /><span class="token instruction"><span class="token keyword">COPY</span> package-lock.json /code/package-lock.json</span><br /><br /><span class="token instruction"><span class="token keyword">WORKDIR</span> /code</span><br /><span class="token instruction"><span class="token keyword">RUN</span> npm i</span><br /><br /><span class="token instruction"><span class="token keyword">COPY</span> . /code</span><br /><br /><span class="token instruction"><span class="token keyword">CMD</span> npm start</span></code></pre>
<p>By doing this, you will not have to do an <code>npm install</code> unless the files <code>package.json</code> or <code>package-lock.json</code> changes.
Any other file in the current directory can change but docker will not rebuild everything, it will just rebuild from copying <code>/code</code>
part.</p>
Publishing a console application to pypi2018-12-28T00:00:00Zhttps://blog.meain.io/2018/publish-console-application-pypi/<p>Let's say you have a very useful python script that you use and think other people might have some use out of it.</p>
<p>One way is to just share the python snippet, but nah, that is too old school.
Let us publish it to pypi so that anyone can just do a <code>pip install your-awsome-tool</code> and get going with it.</p>
<h1 id="so%2C-how-to-do-it%3F" tabindex="-1">So, how to do it? <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#so%2C-how-to-do-it%3F">#</a></h1>
<p>Well, let us create a simple project. Maybe a simple countdown timer.</p>
<blockquote>
<p>I am gonna call the project <code>counterer</code> because I am too lazy to come up with a better name.</p>
</blockquote>
<p>Here is the initial project structure.</p>
<pre><code>.
├── README.md
├── counterer
│ └── counterer.py
└── setup.py
</code></pre>
<p>Pretty simple, the main work comes in <code>setup.py</code>.</p>
<h2 id="create-our-counter-application" tabindex="-1">Create our <code>counter</code> application <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#create-our-counter-application">#</a></h2>
<p>Here is a sample <code>counterer.py</code>.</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">import</span> time<br /><span class="token keyword">import</span> sys<br /><br /><br /><span class="token keyword">def</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span><br /> count <span class="token operator">=</span> <span class="token builtin">int</span><span class="token punctuation">(</span>sys<span class="token punctuation">.</span>argv<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">)</span><br /> <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">reversed</span><span class="token punctuation">(</span><span class="token builtin">range</span><span class="token punctuation">(</span>count<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token keyword">print</span><span class="token punctuation">(</span>i<span class="token punctuation">)</span><br /> time<span class="token punctuation">.</span>sleep<span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span><br /><br /><br /><span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">"__main__"</span><span class="token punctuation">:</span><br /> main<span class="token punctuation">(</span><span class="token punctuation">)</span></code></pre>
<h2 id="configuring-setup.py-file" tabindex="-1">Configuring <code>setup.py</code> file <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#configuring-setup.py-file">#</a></h2>
<p>There is a sample file at the bottom of the blog.</p>
<h3 id="setting-up-a-description-for-pypi-project-page" tabindex="-1">Setting up a description for <code>pypi</code> project page <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#setting-up-a-description-for-pypi-project-page">#</a></h3>
<p>We can use the <code>README.md</code> of our project as the description (<code>long_description</code>) of out project.
For that just use python to read the file and use it.
You have to set two values in <code>setuptools.setup</code></p>
<p>The values to be set are:</p>
<ul>
<li><code>long_description</code>: read the contents of the <code>README.md</code> file and set that</li>
<li><code>long_description_content_type='text/markdown'</code>: set this if you are using markdown, by default it is <code>rst</code></li>
</ul>
<h2 id="setting-up-classifiers" tabindex="-1">Setting up <code>classifiers</code> <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#setting-up-classifiers">#</a></h2>
<p>Pypi uses classifiers as a way to tag projects, it is passed in as the value <code>classifiers</code> in <code>setuptools.setup</code>
function.
Set the classifiers you need as a list.
More info and a list of classifiers can be found at <a href="https://pypi.org/classifiers/">https://pypi.org/classifiers/</a>.</p>
<h2 id="providing-requirements-for-project" tabindex="-1">Providing requirements for project <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#providing-requirements-for-project">#</a></h2>
<p>You can provide the requirements of your project in <code>install_requires</code>. It is just a list of strings.</p>
<p>You could either provide it manually here or just read it from the <code>requirements.txt</code> in your project.</p>
<p>It will be something like.</p>
<pre class="language-python"><code class="language-python">install_requires <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token string">"psutil"</span><span class="token punctuation">,</span> <span class="token string">"pprint"</span><span class="token punctuation">]</span></code></pre>
<p>You pass that on to <code>setuptools.setup</code> as <code>install_requires</code></p>
<h2 id="specifying-entry_points" tabindex="-1">Specifying <code>entry_points</code> <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#specifying-entry_points">#</a></h2>
<p>This is where we define the name of our cli application in <code>setuptools.setup</code>.</p>
<p>In our case, the value will be</p>
<pre class="language-json"><code class="language-json"><span class="token punctuation">{</span> <span class="token property">"console_scripts"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"counter = counterer.counterer:main"</span><span class="token punctuation">]</span> <span class="token punctuation">}</span></code></pre>
<p>With this defined, you can use the command <code>counter</code> and python will call the <code>main</code> function in the <code>counterer</code> package
in the <code>counterer</code> project.</p>
<h2 id="calling-setuptools.setup" tabindex="-1">Calling <code>setuptools.setup</code> <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#calling-setuptools.setup">#</a></h2>
<p>Finally you can call <code>setuptools.setup</code> will all the values.</p>
<p>A few other values that you have to pass that I did not mention are</p>
<ul>
<li><code>packages</code>: list the packages in your project. In our we just have <code>counterer</code></li>
<li><code>name</code>: project name</li>
<li><code>version</code>: version of your project</li>
<li><code>author</code>: your (maintainer's) name</li>
<li><code>author_email</code>: your (maintainer's) email</li>
<li><code>keywords</code>: keywords related to your project</li>
</ul>
<h2 id="a-sample" tabindex="-1">A sample <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#a-sample">#</a></h2>
<p>Let us see what a sample <code>setup.py</code> file looks like.</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">import</span> setuptools<br /><br /><span class="token keyword">with</span> <span class="token builtin">open</span><span class="token punctuation">(</span><span class="token string">"README.md"</span><span class="token punctuation">,</span> <span class="token string">"r"</span><span class="token punctuation">)</span> <span class="token keyword">as</span> fh<span class="token punctuation">:</span> <span class="token comment"># description to be used in pypi project page</span><br /> long_description <span class="token operator">=</span> fh<span class="token punctuation">.</span>read<span class="token punctuation">(</span><span class="token punctuation">)</span><br /><br />classifiers <span class="token operator">=</span> <span class="token punctuation">[</span><br /> <span class="token string">"Programming Language :: Python :: 3"</span><span class="token punctuation">,</span><br /> <span class="token string">"License :: OSI Approved :: MIT License"</span><span class="token punctuation">,</span><br /> <span class="token string">"Operating System :: OS Independent"</span><span class="token punctuation">,</span><br /><span class="token punctuation">]</span><br /><br />install_requires <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span> <span class="token comment"># any requirements your package has</span><br /><br />setuptools<span class="token punctuation">.</span>setup<span class="token punctuation">(</span><br /> name<span class="token operator">=</span><span class="token string">"counterer"</span><span class="token punctuation">,</span><br /> version<span class="token operator">=</span><span class="token string">"0.0.1"</span><span class="token punctuation">,</span><br /> author<span class="token operator">=</span><span class="token string">"Abin Simon"</span><span class="token punctuation">,</span><br /> author_email<span class="token operator">=</span><span class="token string">"abinsimon10@gmail.com"</span><span class="token punctuation">,</span><br /> description<span class="token operator">=</span><span class="token string">"Simple counter"</span><span class="token punctuation">,</span><br /> url<span class="token operator">=</span><span class="token string">"https://github.com/meain/counterer"</span><span class="token punctuation">,</span><br /> long_description<span class="token operator">=</span>long_description<span class="token punctuation">,</span><br /> long_description_content_type<span class="token operator">=</span><span class="token string">'text/markdown'</span><span class="token punctuation">,</span><br /> packages<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"counterer"</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> install_requires<span class="token operator">=</span>install_requires<span class="token punctuation">,</span><br /> keywords<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"counter"</span><span class="token punctuation">,</span> <span class="token string">"python"</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> classifiers<span class="token operator">=</span>classifiers<span class="token punctuation">,</span><br /> entry_points<span class="token operator">=</span><span class="token punctuation">{</span><span class="token string">"console_scripts"</span><span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token string">"counter = counterer.counterer:main"</span><span class="token punctuation">]</span><span class="token punctuation">}</span><span class="token punctuation">,</span><br /><span class="token punctuation">)</span></code></pre>
<h1 id="publishing-to-pypi" tabindex="-1">Publishing to Pypi <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#publishing-to-pypi">#</a></h1>
<p>Now that we have our project ready, we just have to publish it.
Register for an account if you do not have one at <a href="https://pypi.org/account/register/">https://pypi.org/account/register/</a>.</p>
<p>Run the following commands to publish it to pypi.</p>
<ul>
<li><code>python3 setup.py sdist bdist_wheel</code></li>
<li><code>twine upload dist/*</code></li>
</ul>
<p>More info about it at <a href="https://pypi.org/project/twine/">https://pypi.org/project/twine/</a>.</p>
<h1 id="aaaand-we-are-done" tabindex="-1">Aaaand we are done <a class="direct-link" href="https://blog.meain.io/2018/publish-console-application-pypi/#aaaand-we-are-done">#</a></h1>
<blockquote>
<p>You can check out the code <a href="https://github.com/meain/counterer">here</a>.</p>
</blockquote>
<p>There you go, now anyone can install your package by using</p>
<p><code>pip3 install counterer</code></p>
<p>After installation they can call</p>
<pre><code>counter 10
</code></pre>
<p>to set a countdown timer for 10 seconds.</p>
A really simple nginx config2019-01-08T00:00:00Zhttps://blog.meain.io/2019/a-really-simple-nginx-conf/<p>This is another installment of "this is not a blog but a note for my future reference".</p>
<p>Here, I will introduce you to writing a very simple <code>nginx</code> config file.
The use case I will go over here assumes you have a directory of static files and a backend running somewhere.</p>
<h1 id="where-to-put-the-files" tabindex="-1">Where to put the files <a class="direct-link" href="https://blog.meain.io/2019/a-really-simple-nginx-conf/#where-to-put-the-files">#</a></h1>
<p>First of all the config file is supposed to be located at <code>/etc/nginx/sites-available/yourapp</code>.
Here is a <a href="https://stackoverflow.com/questions/11693135/multiple-websites-on-nginx-sites-available">stackoverflow link</a>
to what is <code>sites-available</code>. You will have to have symlink it to <code>sites-enabled</code>.</p>
<p>You can actually write the config at <code>/etc/nginx/nginx.conf</code> but you just
create(you will probably have one) a base config there and include this file there.
The <code>/etc/nginx/nginx.conf</code> file will have(need) something like this inside the <code>http</code> block.</p>
<pre><code>include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
</code></pre>
<h1 id="what-to-put-in-the-files" tabindex="-1">What to put in the files <a class="direct-link" href="https://blog.meain.io/2019/a-really-simple-nginx-conf/#what-to-put-in-the-files">#</a></h1>
<p>OK, now to write a really simple <code>nginx</code> config file.
Below will be something that you will end up with.</p>
<p>A great guide to static file serving and stuff is given <a href="https://docs.nginx.com/nginx/admin-guide/web-server/serving-static-content/">here</a>.</p>
<pre><code>server {
listen 80;
client_max_body_size 200M;
server_name yoursite.com www.yoursite.com;
root /home/username/ui/dist;
index index.html;
location / {
try_files $uri @backend;
}
location @backend {
proxy_pass http://localhost:8080;
}
}
</code></pre>
<p>So, you write the whole thing inside.</p>
<pre><code>server {
-- STUFF --
}
</code></pre>
<p>Btw, you might see code blocks which have <code>server</code> wrapped inside <code>http</code>.
You can skip that we have <code>http</code> inside <code>nginx.conf</code> and we are including this file inside of it.
Maybe a better explanation on <a href="https://stackoverflow.com/questions/20639568/when-do-we-need-to-use-http-block-in-nginx-config-file">stackoverflow</a>.</p>
<h3 id="listen" tabindex="-1"><code>listen</code> <a class="direct-link" href="https://blog.meain.io/2019/a-really-simple-nginx-conf/#listen">#</a></h3>
<p>If it ain't obvious by now, it says <code>nginx</code> has to listen to port 80 for requests.</p>
<h3 id="client_max_body_size" tabindex="-1"><code>client_max_body_size</code> <a class="direct-link" href="https://blog.meain.io/2019/a-really-simple-nginx-conf/#client_max_body_size">#</a></h3>
<p>Define the max size of the content that the client can send. Duh!</p>
<h3 id="server_name" tabindex="-1"><code>server_name</code> <a class="direct-link" href="https://blog.meain.io/2019/a-really-simple-nginx-conf/#server_name">#</a></h3>
<p>Define the name of your server. You use this to say where the requests will come from.
Each server block will have one of these.</p>
<p>So that if you have a subdomain that you would like to map to a completely different project you could do it here.
A quick googling lead me <a href="https://www.digitalocean.com/community/questions/what-exactly-is-server_name-in-nginx-configuration-file">here</a>.</p>
<h3 id="root-and-index" tabindex="-1"><code>root and index</code> <a class="direct-link" href="https://blog.meain.io/2019/a-really-simple-nginx-conf/#root-and-index">#</a></h3>
<p><code>root</code> to specify where to look for static files. Give the path to where your static files are.</p>
<p><code>index</code> is defined so as to say where to look for when there is no file defined, as in which is your <code>index.html</code>
equivalent.</p>
<h3 id="location-%40backend" tabindex="-1"><code>location @backend</code> <a class="direct-link" href="https://blog.meain.io/2019/a-really-simple-nginx-conf/#location-%40backend">#</a></h3>
<p>This is used to tell <code>nginx</code> where your backend is. This could be a local link or even a link to an external server.</p>
<p>With <code>proxy_pass http://localhost:8080;</code> we tell that our backend is at <code>localhost:8080</code>.</p>
<h3 id="location-%2F" tabindex="-1"><code>location /</code> <a class="direct-link" href="https://blog.meain.io/2019/a-really-simple-nginx-conf/#location-%2F">#</a></h3>
<p>The final piece, we tell <code>nginx</code> when a request comes in. You could have multiple <code>location</code> blocks if you have a
specific endpoint like <code>/api</code> or something for your backend.</p>
<p>Here we check if a file like that exists in static directory that we gave, if yes then serve the static file.
Otherwise route it to the backend.</p>
<h1 id="updating-changes-in-nginx" tabindex="-1">Updating changes in nginx <a class="direct-link" href="https://blog.meain.io/2019/a-really-simple-nginx-conf/#updating-changes-in-nginx">#</a></h1>
<p>Once you have changed the nginx config files, you will have to update running nginx instance.
You can do the by doing:</p>
<pre class="language-shell"><code class="language-shell">nginx <span class="token parameter variable">-t</span> <span class="token comment"># check if config looks OK</span><br />systemctl reload nginx</code></pre>
<h1 id="starting-and-stopping-nginx" tabindex="-1">Starting and stopping <code>nginx</code> <a class="direct-link" href="https://blog.meain.io/2019/a-really-simple-nginx-conf/#starting-and-stopping-nginx">#</a></h1>
<p>Well, usually you will have <code>systemctl</code> running <code>nginx</code>, in which case you have the following commands.</p>
<pre class="language-shell"><code class="language-shell">systemctl status nginx <span class="token comment"># check status</span><br />systemctl start nginx <span class="token comment"># start nginx</span><br />systemctl stop nginx <span class="token comment"># stop nginx</span><br />systemctl restart nginx <span class="token comment"># stop then start nginx</span></code></pre>
Understanding Rust macros2019-02-17T00:00:00Zhttps://blog.meain.io/2019/understanding-rust-macros/<p>Yo, I have been working on Rust for some time now. It is a great language and a refreshment coming from a primarily
Python and Javascript background. I feel like the compiler has got my back even though it yells at me a lot. I feel
like I could refactor something and if the compiler does not complain, nothing will break.</p>
<p>But that said, coming from a background of mostly just dynamic languages I had to learn a lot of new stuff. One of those
is macros. They are a really powerful tool once you get a hang of it even though it might look a bit intimidating at
first.</p>
<h1 id="what-are-macros%3F" tabindex="-1">What are <code>macros</code>? <a class="direct-link" href="https://blog.meain.io/2019/understanding-rust-macros/#what-are-macros%3F">#</a></h1>
<p>Well, in short they are "things" that take code at compile time and generate code.
You might get a better understanding of what they are when you actually see something if you have not grasped the idea
already.</p>
<h1 id="why-use-them%3F" tabindex="-1">Why use them? <a class="direct-link" href="https://blog.meain.io/2019/understanding-rust-macros/#why-use-them%3F">#</a></h1>
<p>Macros are a great way to provide simple enough abstractions without much of a performance hit as the transformation to
the necessary code is done during the compilation time.</p>
<h1 id="rust-macro-system" tabindex="-1">Rust macro system <a class="direct-link" href="https://blog.meain.io/2019/understanding-rust-macros/#rust-macro-system">#</a></h1>
<p>Rust has two types of macros.</p>
<ul>
<li>Macros</li>
<li>Procedural Macros</li>
</ul>
<p>I will give a rough idea on what each of them is in the following sections,
but here is the <a href="https://doc.rust-lang.org/book/ch19-06-macros.html?highlight=rust,macros#how-to-write-a-custom--derive--macro">Rust docs</a>
on macros in Rust. They actually do a pretty good job of explaining stuff, I am just putting it out there as a different
way of putting stuff.</p>
<h2 id="macros" tabindex="-1">Macros <a class="direct-link" href="https://blog.meain.io/2019/understanding-rust-macros/#macros">#</a></h2>
<p>Well, these are the simpler macro system, as in <code>proc-macros</code> are the more complex ones.</p>
<p>In this, the idea is you get a set of tokens. You can match on these tokens that you receive like doing a regex match
and then perform operations based on the result that you receive. You can think of it kinda like taking the output of a
lexer and doing something on it.</p>
<h3 id="maybe-a-simple-example%3F" tabindex="-1">Maybe a simple example? <a class="direct-link" href="https://blog.meain.io/2019/understanding-rust-macros/#maybe-a-simple-example%3F">#</a></h3>
<p>Here is a simple example. A macro to square a value.</p>
<pre><code>macro_rules! square {
($val:expr) => { $val*2 }
}
</code></pre>
<p>If you see, it is like a Rust match statement.</p>
<p>You are essentially mapping from <code>$val</code> to <code>$val*2</code>.</p>
<p>Actually a macro definition is like a match statement. You can have multiple paths for it to pick based on what the
input is.</p>
<p>Let us see something which has two paths as an example.</p>
<pre class="language-rust"><code class="language-rust"><span class="token attribute attr-name">#[macro_export]</span><br /><span class="token macro property">macro_rules!</span> pow <span class="token punctuation">{</span><br /> <span class="token punctuation">(</span><span class="token variable">$val</span><span class="token punctuation">:</span><span class="token fragment-specifier punctuation">expr</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token variable">$val</span><span class="token operator">*</span><span class="token number">2</span> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /> <span class="token punctuation">(</span><span class="token variable">$val1</span><span class="token punctuation">:</span><span class="token fragment-specifier punctuation">expr</span><span class="token punctuation">,</span> <span class="token variable">$val2</span><span class="token punctuation">:</span><span class="token fragment-specifier punctuation">expr</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token variable">$val1</span><span class="token punctuation">.</span><span class="token function">pow</span><span class="token punctuation">(</span><span class="token variable">$val2</span><span class="token punctuation">)</span> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Here, with the macro <code>pow</code> if we only pass in one value, it will give the square. But if we pass two arguments, it will
give you the value which is equal to the first value raised to the power of the second value.</p>
<p>Hmm, what happens if I pass in 3 values...? Well, the code won't compile and you get a friendly error warning.</p>
<pre><code>error: no rules expected the token `,`
--> src/main.rs:18:35
|
7 | macro_rules! pow {
| ---------------- when calling this macro
...
18 | println!("{:#?}", pow!{4,4,5});
| ^ no rules expected this token in macro call
error: aborting due to previous error
</code></pre>
<p>What if I need to use an dynamic number of args? Well, checkout the next example.</p>
<h3 id="a-bit-more-involved-example" tabindex="-1">A bit more involved example <a class="direct-link" href="https://blog.meain.io/2019/understanding-rust-macros/#a-bit-more-involved-example">#</a></h3>
<p>So, if you have used <code>vec!</code> before this is how it looks:</p>
<pre class="language-rust"><code class="language-rust"><span class="token keyword">let</span> v<span class="token punctuation">:</span> <span class="token class-name">Vec</span><span class="token operator"><</span><span class="token keyword">u32</span><span class="token operator">></span> <span class="token operator">=</span> <span class="token macro property">vec!</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">,</span> <span class="token number">3</span><span class="token punctuation">]</span><span class="token punctuation">;</span></code></pre>
<p>This creates the variable ( or constant ) <code>v</code> as a vec of three elements.
<strong>Btw, a macro can be differentiated from a function with the <code>!</code> at the end</strong>.</p>
<p>What you are doing here is passing <code>1, 2, 3</code> as an argument to the macro <code>vec</code>. Btw, you could have just as easily used</p>
<pre class="language-rust"><code class="language-rust"><span class="token keyword">let</span> v<span class="token punctuation">:</span> <span class="token class-name">Vec</span><span class="token operator"><</span><span class="token keyword">u32</span><span class="token operator">></span> <span class="token operator">=</span> <span class="token macro property">vec!</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">,</span> <span class="token number">3</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token keyword">let</span> v<span class="token punctuation">:</span> <span class="token class-name">Vec</span><span class="token operator"><</span><span class="token keyword">u32</span><span class="token operator">></span> <span class="token operator">=</span> <span class="token macro property">vec!</span><span class="token punctuation">{</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">,</span> <span class="token number">3</span><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre>
<p>Either of these bracket options work too, it does not matter. It is just that <code>[]</code> feels a bit more appropriate when
creating a <code>vec</code>.</p>
<p>The actual implementation of <code>vec!</code> is <a href="https://github.com/rust-lang/rust/blob/8af675a07576940ba24e3d91abd10b029b937946/src/liballoc/macros.rs#L39">here</a>.
But, for the purpose of explaining stuff let us look a simpler example.</p>
<pre class="language-rust"><code class="language-rust"><span class="token attribute attr-name">#[macro_export]</span><br /><span class="token macro property">macro_rules!</span> vec <span class="token punctuation">{</span><br /> <span class="token punctuation">(</span> $<span class="token punctuation">(</span> <span class="token variable">$x</span><span class="token punctuation">:</span><span class="token fragment-specifier punctuation">expr</span> <span class="token punctuation">)</span><span class="token punctuation">,</span><span class="token operator">*</span> <span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> <span class="token keyword">mut</span> temp_vec <span class="token operator">=</span> <span class="token class-name">Vec</span><span class="token punctuation">::</span><span class="token function">new</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> $<span class="token punctuation">(</span><br /> temp_vec<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span><span class="token variable">$x</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">)</span><span class="token operator">*</span><br /> temp_vec<br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Hmm, that code is something. OK. Let us break down what exactly is happening in there.</p>
<p>So, what happens is that a <em>regexish</em> match on the argument that gets passed into the <code>vec</code> macro is done. This happens
in line 3.</p>
<p>The line essentially checks for expressions that is separated by a comma and assign each of them to a variable <code>x</code>.
Now in lines 6-8 we loop of the values that come into <code>x</code>.</p>
<p><a href="https://doc.rust-lang.org/1.7.0/reference.html#macro-by-example">Here</a> is a list of things that you can use instead of
<code>expr</code>. Or just check here.</p>
<ul>
<li><code>ident</code>: an identifier. Examples: x; foo.</li>
<li><code>path</code>: a qualified name. Example: T::SpecialA.</li>
<li><code>expr</code>: an expression. Examples: 2 + 2; if true { 1 } else { 2 }; f(42).</li>
<li><code>ty</code>: a type. Examples: i32; Vec<(char, String)>; &T.</li>
<li><code>pat</code>: a pattern. Examples: Some(t); (17, 'a'); _.</li>
<li><code>stmt</code>: a single statement. Example: let x = 3.</li>
<li><code>block</code>: a brace-delimited sequence of statements. Example: { log(error, "hi"); return 12; }.</li>
<li><code>item</code>: an item. Examples: fn foo() { }; struct Bar;.</li>
<li><code>meta</code>: a "meta item", as found in attributes. Example: cfg(target_os = "windows").</li>
<li><code>tt</code>: a single token tree.</li>
</ul>
<p>So, essentially what this macro does is that it takes all the expressions that are there and loop over them and does</p>
<pre class="language-rust"><code class="language-rust">temp_vec<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span><span class="token variable">$x</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>You can view it as a mapping between <code>$( $x:expr ),*</code> to <code>$(temp_vec.push($x);)*</code> with extra steps before and a return
of the final result with line 9.</p>
<p>Simple enough, right?</p>
<p>The <code>vec!</code> macro used here will yield something like this:</p>
<pre class="language-rust"><code class="language-rust"><span class="token keyword">let</span> <span class="token keyword">mut</span> temp_vec <span class="token operator">=</span> <span class="token class-name">Vec</span><span class="token punctuation">::</span><span class="token function">new</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />temp_vec<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />temp_vec<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span><span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />temp_vec<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span><span class="token number">3</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />temp_vec</code></pre>
<p>This code is generated during the compile time and will be replaced in the place of <code>vec!</code>. Pretty sweet, right?</p>
<blockquote>
<p>Btw, you can actually recursively call macros, as in the expansion of one macro, you could use another macro and it
will recursively expand them.</p>
</blockquote>
<h1 id="procedural-macros" tabindex="-1">Procedural Macros <a class="direct-link" href="https://blog.meain.io/2019/understanding-rust-macros/#procedural-macros">#</a></h1>
<p>I initially thought of not writing about procedural macros but then I thought I would give a rough intro here. I might
end up writing another blog just about procedural macros later.</p>
<p>Well, this is like a big brother to simple macros. You know, the big guns. The big daddy. The big boss. OK, I am gonna
stop there.
Let us look into what it is.</p>
<blockquote>
<p><strong>The code for the example here is available at <a href="https://github.com/meain/rust-macros-example">meain/rust-macros-example</a></strong></p>
</blockquote>
<p>Procedural macros take some Rust code as input and changes it to some other Rust code. You can more or less think of it
as receiving the AST and modifying it to a different one, kinda like what <code>babel</code> does in the Javascript world but this
is not to make it compatible with an older version or something.</p>
<blockquote>
<p>One small but important thing about proc macros is that they have to reside in their own crate with a specific crate type</p>
</blockquote>
<p>So, a simple macro would have a folder structure something like</p>
<pre><code>.
├── hello_macro
│ ├── Cargo.lock
│ ├── Cargo.toml
│ ├── hello_macro_derive
│ │ ├── Cargo.lock
│ │ ├── Cargo.toml
│ │ └── src
│ ├── src
│ │ └── lib.rs
└── mycode
├── Cargo.lock
├── Cargo.toml
└── src
└── main.rs
</code></pre>
<p>Your code will reside in <code>mycode</code> and the macro in <code>hello_macro</code> folder.</p>
<p>OK, now lets see what is in each of the files.</p>
<p>The <code>Cargo.toml</code> file under <code>hello_macro</code> has nothing fancy. In fact here is what is there in mine</p>
<pre class="language-toml"><code class="language-toml"><span class="token punctuation">[</span><span class="token table class-name">package</span><span class="token punctuation">]</span><br /><span class="token key property">name</span> <span class="token punctuation">=</span> <span class="token string">"hello_macro"</span><br /><span class="token key property">version</span> <span class="token punctuation">=</span> <span class="token string">"0.1.0"</span><br /><span class="token key property">authors</span> <span class="token punctuation">=</span> <span class="token punctuation">[</span><span class="token string">"Abin Simon <abinsimon10@gmail.com>"</span><span class="token punctuation">]</span><br /><span class="token key property">edition</span> <span class="token punctuation">=</span> <span class="token string">"2018"</span><br /><br /><span class="token punctuation">[</span><span class="token table class-name">dependencies</span><span class="token punctuation">]</span></code></pre>
<p>But inside the <code>Cargo.toml</code> file inside <code>hello_macro_derive</code> we have some stuff. Let me show you what is in there and I
will go over what they are</p>
<pre class="language-toml"><code class="language-toml"><span class="token punctuation">[</span><span class="token table class-name">package</span><span class="token punctuation">]</span><br /><span class="token key property">name</span> <span class="token punctuation">=</span> <span class="token string">"hello_macro_derive"</span><br /><span class="token key property">version</span> <span class="token punctuation">=</span> <span class="token string">"0.1.0"</span><br /><span class="token key property">authors</span> <span class="token punctuation">=</span> <span class="token punctuation">[</span><span class="token string">"Abin Simon <abinsimon10@gmail.com>"</span><span class="token punctuation">]</span><br /><span class="token key property">edition</span> <span class="token punctuation">=</span> <span class="token string">"2018"</span><br /><br /><span class="token punctuation">[</span><span class="token table class-name">lib</span><span class="token punctuation">]</span><br /><span class="token key property">proc-macro</span> <span class="token punctuation">=</span> <span class="token boolean">true</span><br /><br /><span class="token punctuation">[</span><span class="token table class-name">dependencies</span><span class="token punctuation">]</span><br /><span class="token key property">syn</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span> <span class="token key property">version</span><span class="token punctuation">=</span><span class="token string">"0.15"</span><span class="token punctuation">,</span> <span class="token key property">features</span><span class="token punctuation">=</span><span class="token punctuation">[</span><span class="token string">"extra-traits"</span><span class="token punctuation">]</span> <span class="token punctuation">}</span><br /><span class="token key property">quote</span> <span class="token punctuation">=</span> <span class="token string">"0.6"</span></code></pre>
<p>One thing you have to do here is under the lib section you have to specify that this is a <code>proc-macro</code>.</p>
<p>Now, the deps. There are two basic deps that you will end up needing.</p>
<p>If you see, rust will give you all the stuff that is passed into the macro as a stream of tokens.
The <a href="https://github.com/dtolnay/syn">syn</a> crate will help you parse that and change it into something which we can
easily work with. Oh btw, I have enable a specific feature of the <code>syn</code> crate. This one is for some debugging purposes.
You can check out other optional features <a href="https://github.com/dtolnay/syn#optional-features">here</a>.</p>
<p>The other dependency that you see is <a href="https://github.com/dtolnay/quote">quote</a>. Now this is kinda does the reverse of
what syn does(not exactly). It helps you write actual Rust code and convert that into a <code>TokenStream</code> which is something
that rust expects out of a macro. If you see quote in itself is a macro. ¯\<em>(ツ)</em>/¯.</p>
<p>Let me get over the basic files first and I will go into how to actually write one.</p>
<p>The file <code>hello__macro/src/lib.rs</code> will look something like this</p>
<pre class="language-rust"><code class="language-rust"><span class="token keyword">pub</span> <span class="token keyword">trait</span> <span class="token type-definition class-name">HelloMacro</span> <span class="token punctuation">{</span><br /> <span class="token keyword">fn</span> <span class="token function-definition function">helpify</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>and the file <code>mycode/src/main.rs</code> ( the one that consumes the macro ) will look something like this.</p>
<pre class="language-rust"><code class="language-rust"><span class="token keyword">use</span> <span class="token namespace">hello_macro<span class="token punctuation">::</span></span><span class="token class-name">HelloMacro</span><span class="token punctuation">;</span><br /><span class="token keyword">use</span> <span class="token namespace">hello_macro_derive<span class="token punctuation">::</span></span><span class="token class-name">HelloMacro</span><span class="token punctuation">;</span><br /><br /><span class="token attribute attr-name">#[allow(dead_code)]</span><br /><span class="token attribute attr-name">#[derive(HelloMacro)]</span><br /><span class="token keyword">struct</span> <span class="token type-definition class-name">Pancakes</span> <span class="token punctuation">{</span><br /> doable<span class="token punctuation">:</span> <span class="token keyword">bool</span><span class="token punctuation">,</span><br /> name<span class="token punctuation">:</span> <span class="token class-name">String</span><span class="token punctuation">,</span><br /> age<span class="token punctuation">:</span> <span class="token keyword">u32</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">fn</span> <span class="token function-definition function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token class-name">Pancakes</span><span class="token punctuation">::</span><span class="token function">helpify</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>OK, with that out of the way, let us look into how the actual macro is written
The file with the macro definition looks something like this</p>
<pre class="language-rust"><code class="language-rust"><span class="token keyword">extern</span> <span class="token keyword">crate</span> <span class="token module-declaration namespace">proc_macro</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">use</span> <span class="token keyword">crate</span><span class="token module-declaration namespace"><span class="token punctuation">::</span>proc_macro<span class="token punctuation">::</span></span><span class="token class-name">TokenStream</span><span class="token punctuation">;</span><br /><span class="token keyword">use</span> <span class="token namespace">quote<span class="token punctuation">::</span></span>quote<span class="token punctuation">;</span><br /><span class="token keyword">use</span> <span class="token namespace">syn<span class="token punctuation">::</span></span><span class="token class-name">Data</span><span class="token punctuation">::</span><span class="token class-name">Struct</span><span class="token punctuation">;</span><br /><span class="token keyword">use</span> <span class="token namespace">syn<span class="token punctuation">::</span></span><span class="token class-name">Fields</span><span class="token punctuation">;</span><br /><span class="token keyword">use</span> <span class="token namespace">syn<span class="token punctuation">::</span></span><span class="token class-name">Type</span><span class="token punctuation">::</span><span class="token class-name">Path</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">fn</span> <span class="token function-definition function">impl_hello_macro</span><span class="token punctuation">(</span>ast<span class="token punctuation">:</span> <span class="token operator">&</span><span class="token namespace">syn<span class="token punctuation">::</span></span><span class="token class-name">DeriveInput</span><span class="token punctuation">)</span> <span class="token punctuation">-></span> <span class="token class-name">TokenStream</span> <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> name <span class="token operator">=</span> <span class="token operator">&</span>ast<span class="token punctuation">.</span>ident<span class="token punctuation">;</span><br /> <span class="token keyword">let</span> data <span class="token operator">=</span> <span class="token operator">&</span>ast<span class="token punctuation">.</span>data<span class="token punctuation">;</span><br /> <span class="token comment">// println!("{:#?}", data);</span><br /> <span class="token keyword">let</span> <span class="token keyword">mut</span> defenition <span class="token operator">=</span> <span class="token macro property">format!</span><span class="token punctuation">(</span><span class="token string">"Struct {}"</span><span class="token punctuation">,</span> name<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token class-name">Struct</span><span class="token punctuation">(</span>def<span class="token punctuation">)</span> <span class="token operator">=</span> data <span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token class-name">Fields</span><span class="token punctuation">::</span><span class="token class-name">Named</span><span class="token punctuation">(</span>fields<span class="token punctuation">)</span> <span class="token operator">=</span> <span class="token operator">&</span>def<span class="token punctuation">.</span>fields <span class="token punctuation">{</span><br /> <span class="token keyword">for</span> named <span class="token keyword">in</span> <span class="token operator">&</span>fields<span class="token punctuation">.</span>named <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> ident <span class="token operator">=</span> <span class="token operator">&</span>named<span class="token punctuation">.</span>ident<span class="token punctuation">;</span><br /> <span class="token keyword">let</span> ty <span class="token operator">=</span> <span class="token operator">&</span>named<span class="token punctuation">.</span>ty<span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token class-name">Some</span><span class="token punctuation">(</span>id<span class="token punctuation">)</span> <span class="token operator">=</span> ident <span class="token punctuation">{</span><br /> <span class="token comment">// println!("{}", id);</span><br /> defenition <span class="token operator">=</span> <span class="token macro property">format!</span><span class="token punctuation">(</span><span class="token string">"{}\n {}:"</span><span class="token punctuation">,</span> defenition<span class="token punctuation">,</span> id<span class="token punctuation">)</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token class-name">Path</span><span class="token punctuation">(</span>path<span class="token punctuation">)</span> <span class="token operator">=</span> ty <span class="token punctuation">{</span><br /> <span class="token comment">// println!("{}", path.path.segments[0].ident);</span><br /> defenition <span class="token operator">=</span> <span class="token macro property">format!</span><span class="token punctuation">(</span><span class="token string">"{} {}"</span><span class="token punctuation">,</span> defenition<span class="token punctuation">,</span> path<span class="token punctuation">.</span>path<span class="token punctuation">.</span>segments<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>ident<span class="token punctuation">)</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">let</span> gen <span class="token operator">=</span> <span class="token macro property">quote!</span> <span class="token punctuation">{</span><br /> <span class="token keyword">impl</span> <span class="token class-name">HelloMacro</span> <span class="token keyword">for</span> #name <span class="token punctuation">{</span><br /> <span class="token keyword">fn</span> <span class="token function-definition function">helpify</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token macro property">println!</span><span class="token punctuation">(</span>#defenition<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /> gen<span class="token punctuation">.</span><span class="token function">into</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /><span class="token punctuation">}</span><br /><br /><span class="token attribute attr-name">#[proc_macro_derive(HelloMacro)]</span><br /><span class="token keyword">pub</span> <span class="token keyword">fn</span> <span class="token function-definition function">hello_macro_derive</span><span class="token punctuation">(</span>input<span class="token punctuation">:</span> <span class="token class-name">TokenStream</span><span class="token punctuation">)</span> <span class="token punctuation">-></span> <span class="token class-name">TokenStream</span> <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> ast <span class="token operator">=</span> <span class="token namespace">syn<span class="token punctuation">::</span></span><span class="token function">parse</span><span class="token punctuation">(</span>input<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">impl_hello_macro</span><span class="token punctuation">(</span><span class="token operator">&</span>ast<span class="token punctuation">)</span><br /><span class="token punctuation">}</span></code></pre>
<p>Here if you check the arguments, <code>ast</code> is the thing that you get after your code goes through the syn package.</p>
<p>The code about comprises of two sections. The parsing of the <code>ast</code> and generating a simple string and a part which adds
a func that will display the result onto the <code>impl</code> of the struct.</p>
<p>Let us look into the <code>ast</code> parsing part. If you check the data variable, for this current one it looks something like
this</p>
<pre><code>Struct(
DataStruct {
struct_token: Struct,
fields: Named(
FieldsNamed {
brace_token: Brace,
named: [
Field {
attrs: [],
vis: Inherited,
ident: Some(
Ident {
ident: "doable",
span: #0 bytes(130..136)
}
),
colon_token: Some(
Colon
),
ty: Path(
TypePath {
qself: None,
path: Path {
leading_colon: None,
segments: [
PathSegment {
ident: Ident {
ident: "bool",
span: #0 bytes(138..142)
},
arguments: None
}
]
}
}
)
},
Comma,
Field {
attrs: [],
vis: Inherited,
ident: Some(
Ident {
ident: "name",
span: #0 bytes(148..152)
}
),
colon_token: Some(
Colon
),
ty: Path(
TypePath {
qself: None,
path: Path {
leading_colon: None,
segments: [
PathSegment {
ident: Ident {
ident: "String",
span: #0 bytes(154..160)
},
arguments: None
}
]
}
}
)
},
Comma,
Field {
attrs: [],
vis: Inherited,
ident: Some(
Ident {
ident: "age",
span: #0 bytes(166..169)
}
),
colon_token: Some(
Colon
),
ty: Path(
TypePath {
qself: None,
path: Path {
leading_colon: None,
segments: [
PathSegment {
ident: Ident {
ident: "u32",
span: #0 bytes(171..174)
},
arguments: None
}
]
}
}
)
}
]
}
),
semi_token: None
}
)
</code></pre>
<p>Well, now in the following block, ie</p>
<pre class="language-rust"><code class="language-rust"><span class="token keyword">let</span> <span class="token keyword">mut</span> defenition <span class="token operator">=</span> <span class="token macro property">format!</span><span class="token punctuation">(</span><span class="token string">"Struct {}"</span><span class="token punctuation">,</span> name<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token class-name">Struct</span><span class="token punctuation">(</span>def<span class="token punctuation">)</span> <span class="token operator">=</span> data <span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token class-name">Fields</span><span class="token punctuation">::</span><span class="token class-name">Named</span><span class="token punctuation">(</span>fields<span class="token punctuation">)</span> <span class="token operator">=</span> <span class="token operator">&</span>def<span class="token punctuation">.</span>fields <span class="token punctuation">{</span><br /> <span class="token keyword">for</span> named <span class="token keyword">in</span> <span class="token operator">&</span>fields<span class="token punctuation">.</span>named <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> ident <span class="token operator">=</span> <span class="token operator">&</span>named<span class="token punctuation">.</span>ident<span class="token punctuation">;</span><br /> <span class="token keyword">let</span> ty <span class="token operator">=</span> <span class="token operator">&</span>named<span class="token punctuation">.</span>ty<span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token class-name">Some</span><span class="token punctuation">(</span>id<span class="token punctuation">)</span> <span class="token operator">=</span> ident <span class="token punctuation">{</span><br /> <span class="token comment">// println!("{}", id);</span><br /> defenition <span class="token operator">=</span> <span class="token macro property">format!</span><span class="token punctuation">(</span><span class="token string">"{}\n {}:"</span><span class="token punctuation">,</span> defenition<span class="token punctuation">,</span> id<span class="token punctuation">)</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token class-name">Path</span><span class="token punctuation">(</span>path<span class="token punctuation">)</span> <span class="token operator">=</span> ty <span class="token punctuation">{</span><br /> <span class="token comment">// println!("{}", path.path.segments[0].ident);</span><br /> defenition <span class="token operator">=</span> <span class="token macro property">format!</span><span class="token punctuation">(</span><span class="token string">"{} {}"</span><span class="token punctuation">,</span> defenition<span class="token punctuation">,</span> path<span class="token punctuation">.</span>path<span class="token punctuation">.</span>segments<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>ident<span class="token punctuation">)</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre>
<p>we just create a string in <code>defenition</code> variable. Well, it is just some patchy code to go through the data structure and
I don't think I should go through this. Also you can probably make something better.</p>
<p>Now to the second part, we add the func to the struct. We use the <code>quote</code> package to do this.</p>
<pre class="language-rust"><code class="language-rust"><span class="token keyword">let</span> gen <span class="token operator">=</span> <span class="token macro property">quote!</span> <span class="token punctuation">{</span><br /> <span class="token keyword">impl</span> <span class="token class-name">HelloMacro</span> <span class="token keyword">for</span> #name <span class="token punctuation">{</span><br /> <span class="token keyword">fn</span> <span class="token function-definition function">helpify</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token macro property">println!</span><span class="token punctuation">(</span>#defenition<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br />gen<span class="token punctuation">.</span><span class="token function">into</span><span class="token punctuation">(</span><span class="token punctuation">)</span></code></pre>
<blockquote>
<p>To use any variable we have defined we have to use a <code>#</code> in front. That is why <code>#defenition</code> is defined like that.</p>
</blockquote>
<p>Now this generated code gets returned with <code>gen.into()</code>. And viola, you have a <code>helpify()</code> in anything <code>HelloMacro</code>.</p>
<p>Well, if we run our code now, you get a response which will look something like this:</p>
<pre><code>$ cargo run rust-macros-example/mycode 137d
Finished dev [unoptimized + debuginfo] target(s) in 0.04s
Running `target/debug/mycode`
Struct Pancakes
doable: bool
name: String
age: u32
</code></pre>
<p>Well, this is more or less what I got, I believe this gave you some kind of a intro to rust macros.</p>
How to do server side autocompletion (networking parts)2019-03-26T00:00:00Zhttps://blog.meain.io/2019/simple-server-side-autocomplete/<p>So, recently I had to create an autocompletion for a chat app at work. We did not need anything fancy for the first cut.
Sounded like a simple project. This was in <code>React</code> I initially thought of pulling in an existing library for it, but our
requirements needed some special kind of autocompletion menu.</p>
<p>Let me tell you what I mean by that. We build chatbots and this was the chat input window.
So if we were building something for, lets say <code>Netflix</code>, the completion has to be able to something like this.</p>
<p><img src="https://i.imgur.com/qoPpbus.gif" alt="completion-sample" /></p>
<p><em>Sorry about the jittery gif</em></p>
<p>As in a separate completion for the possible movie titles.</p>
<p>So, I gave up on just taking an existing library because I could not find anything that did what I needed to do.
Initially I built something simple, the one you saw above, and it worked fine for some time. I was loading the whole
possible completions as a <code>json</code> to the frontend and the doing your normal string matching thingy on it.</p>
<p>But one fine morning I got a request to make the completions to be routed through the backend so that it can be more
dynamic in nature, and a few another reasons. And yeah, I had to do that.</p>
<h2 id="initial-doubts" tabindex="-1">Initial doubts <a class="direct-link" href="https://blog.meain.io/2019/simple-server-side-autocomplete/#initial-doubts">#</a></h2>
<p>Initially I was skeptical about routing the completions through the backend as I felt like it will feel very very slow
compared to the current implementation. But then again, <code>Google</code> was doing it and I was said some delay is expected and
they are fine with that.</p>
<p>So I decided to start working on it. Here is my journey making the change.</p>
<h2 id="hit-the-backend-for-each-keystroke" tabindex="-1">Hit the backend for each keystroke <a class="direct-link" href="https://blog.meain.io/2019/simple-server-side-autocomplete/#hit-the-backend-for-each-keystroke">#</a></h2>
<p>I started off with this idea, created a simple python backend written using <code>flask</code> to test this out.
The <code>flask</code> api did nothing more than take the request, find possible completions based on some creiteria and return the
list of possible completions back.</p>
<p>Initially, I was just hitting the backend for each keystroke. It all seemed to be OK until I went to the network tab and
changed the network from <code>online</code> to <code>Slow 3G</code> ( Chrome lets you simulate different network speeds )</p>
<p>The issue was the when you type a long sentence there will be a lot of requests that will be send and the browser has to
wait on all the previous request to be completed before it can get the response to the final(useful) response. This was
made really bad by the fact that I had just once instance of the flask app running. So all the request had to be
processed one by one.</p>
<p>It looked something like this</p>
<p><img src="https://i.imgur.com/5AFICTN.gif" alt="compl-waiting" /></p>
<h2 id="hit-backend-only-if-user-pauses" tabindex="-1">Hit backend only if user pauses <a class="direct-link" href="https://blog.meain.io/2019/simple-server-side-autocomplete/#hit-backend-only-if-user-pauses">#</a></h2>
<p>Well, this was my next idea. Do not send a request for every keystroke, but just when the user stops typing.
So I put the send request in a <code>setTimeout</code> and cancelled the timeout if a new keystroke came in before the previous
timeout had completed. The function looked something like this in code.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">if</span> <span class="token punctuation">(</span>filterTimeout<span class="token punctuation">)</span> <span class="token function">clearTimeout</span><span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">.</span>state<span class="token punctuation">.</span>inputTimer<span class="token punctuation">)</span><span class="token punctuation">;</span><br />filterTimeout <span class="token operator">=</span> <span class="token function">setTimeout</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">updateFilteredCompletions</span><span class="token punctuation">(</span>input<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">400</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>This seemed to be good idea initially, but it still was not good enough for a few reasons. The main issue was that We
had a useless wait before we sent a request to the server for completion. In the code example above, we are pretty
much wasting <code>400ms</code> which could have been used to send and receive the completion info.</p>
<h2 id="cancellable-requests" tabindex="-1">Cancellable requests <a class="direct-link" href="https://blog.meain.io/2019/simple-server-side-autocomplete/#cancellable-requests">#</a></h2>
<p>Searing for a better way to do this, I came across something called cancellable
requests. More info <a href="https://developers.google.com/web/updates/2017/09/abortable-fetch">here</a>.</p>
<p>So, essentially the idea was. You send a request but at a later point if you decide that you do not need the response
you can just cancel the request so that you do not have to bother about receiving it nor does the server has to bother
about sending it.</p>
<p>This works if the majority of time taken is in the transmission rather than the actual computation happening in the
backend. For me this was the case as the computation took just like <code>0.0013s</code> but the whole transfer with simulated 3G
speeds took like 1.5 - 2 seconds. So I decided to go this route.</p>
<p>I decided to add the cancellation thing. It was just awesome.
See the difference that it makes.</p>
<p><img src="https://i.imgur.com/gUxiv9i.gif" alt="completion-with-cancellation" /></p>
<p>It might feel like it is still slow, but see the difference it makes relative to the previous implementation. Also
remember that this is with <code>Slow 3G</code> speeds. If you notice the network tab, you can see that request are getting
cancelled when new one is coming and we are not having a <code>(pending..)</code> sign which causes the browser to wait for its
response and the server to send it. This is more of an issue for the server sending as it might be having just one
thread running to do the completion.</p>
<p>This <a href="https://stackoverflow.com/a/47250621/2724649">StackOverflow answer</a> sums up really well on how to implement
something like this.</p>
<h2 id="a-few-tiny-optimisations" tabindex="-1">A few tiny optimisations <a class="direct-link" href="https://blog.meain.io/2019/simple-server-side-autocomplete/#a-few-tiny-optimisations">#</a></h2>
<p>With the cancellable requests in place, I was mostly happy with the result. But I decided to add in a few more nice
things.</p>
<h4 id="caching-completions" tabindex="-1">Caching completions <a class="direct-link" href="https://blog.meain.io/2019/simple-server-side-autocomplete/#caching-completions">#</a></h4>
<p>One simple idea was to cache completions. The idea is simple, you just maintain a global dict with <code>input</code> ->
<code>completion</code> mapping. Also, remember to do cache invalidation properly.</p>
<p>You might think "why would anyone cache the autocompletion result", but the main use of this comes when the user presses
the backspace key. It might feel like a small thing, but it makes a big difference.</p>
<h3 id="trim-out-the-input-before-sending" tabindex="-1">Trim out the input before sending <a class="direct-link" href="https://blog.meain.io/2019/simple-server-side-autocomplete/#trim-out-the-input-before-sending">#</a></h3>
<p>Another very tiny improvement. In most cases the completion for <code>Show me the rating for </code> and <code>Show me the rating for </code>
and <code>Show me the rating for </code> will be the same. So just avoid it ( also handle this in caching ).</p>
<p>The end result of the above things will look something like this</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">if</span> <span class="token punctuation">(</span>input<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">in</span> completionResponses<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> completionResponsesCache<span class="token punctuation">[</span>input<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token comment">// from cache</span><br /><span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> completions <span class="token operator">=</span> <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">getCompletionsFromBackend</span><span class="token punctuation">(</span>input<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// api call ( I am gonna pretend this is a synchronous call )</span><br /> completionResponsesCache<span class="token punctuation">[</span>input<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token operator">=</span> completions<br /> rturn completions<br /><span class="token punctuation">}</span></code></pre>
<p>After that add in a few nice UI touches and you get a not so bad completion experience. I leave that part to you.
And that is a wrap. Thanks for reading my rant.</p>
A bit more about configuring nginx (rewrite and alias)2019-04-30T00:00:00Zhttps://blog.meain.io/2019/a-bit-more-comlex-nginx-config/<p>We went through some basic stuff you need to understand to configure <code>nginx</code> in <a href="https://blog.meain.io/2019/a-really-simple-nginx-conf/">an old blog</a>.
After writing that I had to work on another project which was a bit messy.
So I went over more stuff, and I thought I would write about it.</p>
<p>Before going into the blog, let me go in what is the situation that I was in.</p>
<p>I had two frontend (static file) endpoints and two apis.
Let us call the frontends f1 and f2 and the apis a1 and a2.</p>
<p>f1 is a completely independent one with no backend.
f2 depends on both a1 and a2.</p>
<p>There was some things we did in the frontend.
Request from f2 to a1 was prefixed by <code>/a1</code> which had to be removed before we could process in the api a1.
Same for a2.</p>
<p>Here is the important part of the config ended up using.</p>
<pre class="language-nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">server</span></span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">server_name</span> mysite.com</span><span class="token punctuation">;</span><br /><br /> <span class="token directive"><span class="token keyword">root</span> /home/meain/f1/dist</span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">index</span> index.html</span><span class="token punctuation">;</span><br /><br /> <span class="token directive"><span class="token keyword">location</span> /f2</span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">alias</span> /home/meain/f2/dist</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token directive"><span class="token keyword">location</span> /</span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">try_files</span> <span class="token variable">$uri</span> <span class="token variable">$uri</span>/ =404</span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">expires</span> max</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token directive"><span class="token keyword">location</span> /api2</span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">rewrite</span> ^/api2/(.*)$ /<span class="token variable">$1</span> break</span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">proxy_pass</span> http://localhost:8001</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token directive"><span class="token keyword">location</span> /api1</span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">rewrite</span> /api1/(.*)$ /<span class="token variable">$1</span> last</span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">proxy_pass</span> http://localhost:8000</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>Essentially I just want to introduce you to <code>alias</code> and <code>rewrite</code>.</p>
<p>So, if you check the <code>location /</code> block, it checks for the static file and if it cannot find it, it returns <code>404</code>.
You could check out about returning a custom <code>404</code> page <a href="https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-to-use-custom-error-pages-on-ubuntu-14-04">here</a></p>
<p>For f2, we just change the location where nginx checks for the files by using the <code>alias</code> keyword.</p>
<p>Now for using the backends, we have to remove the extra <code>/api1</code> or <code>/api2</code> in front
For this we make use of the <code>rewrite</code> keyword.</p>
<p>We use <code>rewrite</code> to write a regex that will transform the incoming request url.
Here we are using capture groups to just remove the <code>/api1</code> or <code>/api2</code> in front.</p>
<p>And, guess what, it works.</p>
<p>Well, this is a small one, but I am guessing it came in handy for someone.</p>
Automatically list files after cd2019-05-06T00:00:00Zhttps://blog.meain.io/2019/automatically-ls-after-cd/<p>Almost everybody does an <code>ls</code> after they <code>cd</code> into a folder.
So why not get your shell to automatically do it?</p>
<p align="center">
<img src="https://i.imgur.com/00UyQyj.png" />
</p>
<h3 id="the-simple-way" tabindex="-1">The simple way <a class="direct-link" href="https://blog.meain.io/2019/automatically-ls-after-cd/#the-simple-way">#</a></h3>
<p>Well, you might know the simple way.
Just change <code>cd</code> to do <code>cd</code> and <code>ls</code>.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function-name function">cd</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token builtin class-name">builtin</span> <span class="token builtin class-name">cd</span> <span class="token string">"<span class="token variable">$@</span>"</span><span class="token punctuation">;</span>ll<span class="token punctuation">;</span><span class="token punctuation">}</span></code></pre>
<h3 id="the-grown-up-way-(in-zsh)" tabindex="-1">The grown up way (in <code>zsh</code>) <a class="direct-link" href="https://blog.meain.io/2019/automatically-ls-after-cd/#the-grown-up-way-(in-zsh)">#</a></h3>
<p>Well, even though the previous one works well for most cases, there might be some situations in which it will not work.
The cases in which it might not work is when you don't actually use <code>cd</code> change dir, but some other command.</p>
<p>Here is the code that you will be using. Just add this to your <code>.zshrc</code> and you should be good.</p>
<pre class="language-shell"><code class="language-shell"><span class="token keyword">function</span> <span class="token function-name function">list_all</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> emulate <span class="token parameter variable">-L</span> <span class="token function">zsh</span><br /> <span class="token function">ls</span><br /><span class="token punctuation">}</span><br /><span class="token keyword">if</span> <span class="token punctuation">[</span><span class="token punctuation">[</span> <span class="token variable">${chpwd_functions<span class="token punctuation">[</span>(r)list_all<span class="token punctuation">]</span>}</span> <span class="token operator">!=</span> <span class="token string">"list_all"</span> <span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">;</span><span class="token keyword">then</span><br /> <span class="token assign-left variable">chpwd_functions</span><span class="token operator">=</span><span class="token punctuation">(</span><span class="token variable">${chpwd_functions<span class="token punctuation">[</span>@<span class="token punctuation">]</span>}</span> <span class="token string">"list_all"</span><span class="token punctuation">)</span><br /><span class="token keyword">fi</span></code></pre>
<p>OK, if that looks like some weird mess, don't worry. I'll explain.</p>
<p>So, <code>zsh</code> lets you run some functions after running anything that will change directory.
The commands that are run are listed in this variable called <code>chpwd_functions</code> and you can add more to the list.</p>
<p>So what we do here in the code is create a function(<code>list_all</code> in our case) which just does an ls and add it to the list of commands
that will be run after a change in directory happens.</p>
<p>There is just some check happening to make sure that the function is not already in the list before we add it. That is
what that if condition does.</p>
<p>Well, that is it. Happy <code>cd</code>ing.</p>
Beginners guide to tensorflow serving2019-05-11T00:00:00Zhttps://blog.meain.io/2019/beginner-guide-to-tf-serving/<p><code>tf-serving</code> if you don't know is a tool that Google has built to serve model built using <code>tensorflow</code>.
Even <code>keras</code> models with a <code>tensorflow</code> backend should do just fine.</p>
<p>Even thought there are a lot of guides on how to use <code>tf-serving</code>, I could not find anything coherent and simple.
So I decided to write one, mostly so that next time I have to do this I would have something to refer to.</p>
<h2 id="why-tf-serving" tabindex="-1">Why <code>tf-serving</code> <a class="direct-link" href="https://blog.meain.io/2019/beginner-guide-to-tf-serving/#why-tf-serving">#</a></h2>
<p>You could just put your model behind a simple flask API and that will work pretty fine for small use cases.</p>
<p><code>tf-serving</code> mostly comes in handy you have heavy load. It is also pretty useful when you have to version you models and
have something like a CI - CD pipeline. This <a href="https://www.youtube.com/watch?v=q_IkJcPyNl0">video</a> explains it pretty well.</p>
<h2 id="how-tf-serving" tabindex="-1">How <code>tf-serving</code> <a class="direct-link" href="https://blog.meain.io/2019/beginner-guide-to-tf-serving/#how-tf-serving">#</a></h2>
<p>OK, now let us get to the part why you might be reading this blog for.
We will be using the <code>tensorflow/serving</code> docker container to run the whole thing. This makes things a whole lot
simpler. Also later when you have to put the whole thing behind <code>kubernetes</code> acting like a load balancer you will end
up using it anyway.</p>
<h3 id="folder-structure" tabindex="-1">Folder structure <a class="direct-link" href="https://blog.meain.io/2019/beginner-guide-to-tf-serving/#folder-structure">#</a></h3>
<p><code>tf-serving</code> needs the model files to be in a specific structure. It should look something like this.</p>
<pre><code>models # base folder for all the models
└── mymodel # model name
└── 1 # model version
├── saved_model.pb
└── variables
├── variables.data-00000-of-00001
└── variables.index
</code></pre>
<p>We will have a base folder called <code>models</code> (you could name it anything, but we will have to pass on the same name to
<code>tf-serving</code>).</p>
<p>Inside the base folder we will have different models. The name of the model that I am using here is <code>mymodel</code>, so we
have that as the folder name here.</p>
<p>Inside that we will have folders with names <code>1</code>, <code>2</code>, <code>3</code> ... etc. These will be different version. It is set up like
this so that when you have a new version, you can just add a new folder and <code>tf-serving</code> will automatically switch to
the new model without restarting. Plus you get some form of versioning.</p>
<h3 id="what-goes-inside-them" tabindex="-1">What goes inside them <a class="direct-link" href="https://blog.meain.io/2019/beginner-guide-to-tf-serving/#what-goes-inside-them">#</a></h3>
<p>OK, now that we know where to put the files, let us see what to put in there.</p>
<p><code>tf-serving</code> will need the files to be in a format what it calls <code>SavedModel</code>.
You can find more about it <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/saved_model/README.md">here</a>.</p>
<p>We have utils inside of <code>tensorflow</code> which will let us convert our models into SavedModel.
Here I will show how to do it for a keras model.</p>
<pre class="language-python"><code class="language-python">signature <span class="token operator">=</span> tf<span class="token punctuation">.</span>saved_model<span class="token punctuation">.</span>signature_def_utils<span class="token punctuation">.</span>predict_signature_def<span class="token punctuation">(</span><br /> inputs<span class="token operator">=</span><span class="token punctuation">{</span><span class="token string">"image"</span><span class="token punctuation">:</span> model<span class="token punctuation">.</span><span class="token builtin">input</span><span class="token punctuation">}</span><span class="token punctuation">,</span> outputs<span class="token operator">=</span><span class="token punctuation">{</span><span class="token string">"scores"</span><span class="token punctuation">:</span> model<span class="token punctuation">.</span>output<span class="token punctuation">}</span><br /><span class="token punctuation">)</span><br /><br />builder <span class="token operator">=</span> tf<span class="token punctuation">.</span>saved_model<span class="token punctuation">.</span>builder<span class="token punctuation">.</span>SavedModelBuilder<span class="token punctuation">(</span><span class="token string">"./models/mymodel/1"</span><span class="token punctuation">)</span><br />builder<span class="token punctuation">.</span>add_meta_graph_and_variables<span class="token punctuation">(</span><br /> sess<span class="token operator">=</span>keras<span class="token punctuation">.</span>backend<span class="token punctuation">.</span>get_session<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> tags<span class="token operator">=</span><span class="token punctuation">[</span>tf<span class="token punctuation">.</span>saved_model<span class="token punctuation">.</span>tag_constants<span class="token punctuation">.</span>SERVING<span class="token punctuation">]</span><span class="token punctuation">,</span><br /> signature_def_map<span class="token operator">=</span><span class="token punctuation">{</span><br /> tf<span class="token punctuation">.</span>saved_model<span class="token punctuation">.</span>signature_constants<span class="token punctuation">.</span>DEFAULT_SERVING_SIGNATURE_DEF_KEY<span class="token punctuation">:</span> signature<br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /><span class="token punctuation">)</span><br />builder<span class="token punctuation">.</span>save<span class="token punctuation">(</span><span class="token punctuation">)</span></code></pre>
<p>You could add the code right at the end of something like below and
you should have a model in the path <code>./models/mymodel/1</code> with the above specified dir structure.</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">import</span> tensorflow <span class="token keyword">as</span> tf<br /><span class="token keyword">from</span> tensorflow <span class="token keyword">import</span> keras<br /><span class="token keyword">import</span> numpy <span class="token keyword">as</span> np<br /><br />fashion_mnist <span class="token operator">=</span> keras<span class="token punctuation">.</span>datasets<span class="token punctuation">.</span>fashion_mnist<br /><span class="token punctuation">(</span>train_images<span class="token punctuation">,</span> train_labels<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span>test_images<span class="token punctuation">,</span> test_labels<span class="token punctuation">)</span> <span class="token operator">=</span> fashion_mnist<span class="token punctuation">.</span>load_data<span class="token punctuation">(</span><span class="token punctuation">)</span><br /><br />train_images <span class="token operator">=</span> train_images <span class="token operator">/</span> <span class="token number">255.0</span><br />test_images <span class="token operator">=</span> test_images <span class="token operator">/</span> <span class="token number">255.0</span><br /><br />model <span class="token operator">=</span> keras<span class="token punctuation">.</span>Sequential<span class="token punctuation">(</span><br /> <span class="token punctuation">[</span><br /> keras<span class="token punctuation">.</span>layers<span class="token punctuation">.</span>Flatten<span class="token punctuation">(</span>input_shape<span class="token operator">=</span><span class="token punctuation">(</span><span class="token number">28</span><span class="token punctuation">,</span> <span class="token number">28</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> keras<span class="token punctuation">.</span>layers<span class="token punctuation">.</span>Dense<span class="token punctuation">(</span><span class="token number">128</span><span class="token punctuation">,</span> activation<span class="token operator">=</span>tf<span class="token punctuation">.</span>nn<span class="token punctuation">.</span>relu<span class="token punctuation">)</span><span class="token punctuation">,</span><br /> keras<span class="token punctuation">.</span>layers<span class="token punctuation">.</span>Dense<span class="token punctuation">(</span><span class="token number">10</span><span class="token punctuation">,</span> activation<span class="token operator">=</span>tf<span class="token punctuation">.</span>nn<span class="token punctuation">.</span>softmax<span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token punctuation">]</span><br /><span class="token punctuation">)</span><br /><br />model<span class="token punctuation">.</span><span class="token builtin">compile</span><span class="token punctuation">(</span><br /> optimizer<span class="token operator">=</span><span class="token string">"adam"</span><span class="token punctuation">,</span> loss<span class="token operator">=</span><span class="token string">"sparse_categorical_crossentropy"</span><span class="token punctuation">,</span> metrics<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"accuracy"</span><span class="token punctuation">]</span><br /><span class="token punctuation">)</span><br /><br />model<span class="token punctuation">.</span>fit<span class="token punctuation">(</span>train_images<span class="token punctuation">,</span> train_labels<span class="token punctuation">,</span> epochs<span class="token operator">=</span><span class="token number">5</span><span class="token punctuation">)</span><br />test_loss<span class="token punctuation">,</span> test_acc <span class="token operator">=</span> model<span class="token punctuation">.</span>evaluate<span class="token punctuation">(</span>test_images<span class="token punctuation">,</span> test_labels<span class="token punctuation">)</span><br /><br />predictions <span class="token operator">=</span> model<span class="token punctuation">.</span>predict<span class="token punctuation">(</span>test_images<span class="token punctuation">)</span><br />res <span class="token operator">=</span> np<span class="token punctuation">.</span>argmax<span class="token punctuation">(</span>predictions<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span><br /><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"res:"</span><span class="token punctuation">,</span> res<span class="token punctuation">)</span></code></pre>
<h3 id="running-using-docker" tabindex="-1">Running using <code>docker</code> <a class="direct-link" href="https://blog.meain.io/2019/beginner-guide-to-tf-serving/#running-using-docker">#</a></h3>
<p>Well, i assume you know what docker is. Well if you don't let us think of it as a super lightweight VM (I couldn't be more
wrong when I say lightweight VM, but it is a good analogy). Just install docker from <a href="https://www.docker.com/">here</a>.</p>
<blockquote>
<p>Btw, if you don't know docker, look into it. It is pretty awesome.</p>
</blockquote>
<p>Now you can run something like this.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">docker</span> run <span class="token parameter variable">-t</span> <span class="token parameter variable">--rm</span> <span class="token parameter variable">-p</span> <span class="token number">8501</span>:8501 <span class="token punctuation">\</span><br /> <span class="token parameter variable">-v</span> <span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token builtin class-name">pwd</span><span class="token variable">)</span></span>/models:/models"</span> <span class="token punctuation">\</span><br /> <span class="token parameter variable">-e</span> <span class="token assign-left variable">MODEL_NAME</span><span class="token operator">=</span>mymodel <span class="token punctuation">\</span><br /> tensorflow/serving</code></pre>
<p>OK, what we do here is we use the image <code>tensorflow/serving</code> from <a href="https://hub.docker.com/">Docker Hub</a>.
It is a preconfigured <code>tensorflow</code> serving setup.</p>
<p>The <code>-p</code> option says that we map the <code>8501</code> port of docker to <code>8501</code> port in our local. This is the default <code>REST</code> port
in <code>tf-serving</code>. For <code>gRPC</code> it is <code>8500</code>.</p>
<p>With <code>-v</code> we mount <code>$(pwd)/models</code> to <code>/models</code> inside the container as that is where <code>tf-serving</code> will look for the
files.</p>
<p>Also we specify the <code>MODEL_NAME</code> as <code>mymodel</code> so that <code>tf-serving</code> will run that model.</p>
<h3 id="simple-client" tabindex="-1">Simple client <a class="direct-link" href="https://blog.meain.io/2019/beginner-guide-to-tf-serving/#simple-client">#</a></h3>
<pre class="language-python"><code class="language-python"><span class="token keyword">import</span> json<br /><span class="token keyword">import</span> requests<br /><span class="token keyword">import</span> numpy <span class="token keyword">as</span> np<br /><span class="token keyword">from</span> tensorflow <span class="token keyword">import</span> keras<br /><br />fashion_mnist <span class="token operator">=</span> keras<span class="token punctuation">.</span>datasets<span class="token punctuation">.</span>fashion_mnist<br /><span class="token punctuation">(</span>train_images<span class="token punctuation">,</span> train_labels<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span>test_images<span class="token punctuation">,</span> test_labels<span class="token punctuation">)</span> <span class="token operator">=</span> fashion_mnist<span class="token punctuation">.</span>load_data<span class="token punctuation">(</span><span class="token punctuation">)</span><br />test_images <span class="token operator">=</span> test_images <span class="token operator">/</span> <span class="token number">255.0</span><br /><br />url <span class="token operator">=</span> <span class="token string">'http://localhost:8501/v1/models/mymodel:predict'</span><br />headers <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token string">"content-type"</span><span class="token punctuation">:</span> <span class="token string">"application/json"</span><span class="token punctuation">}</span><br />data <span class="token operator">=</span> json<span class="token punctuation">.</span>dumps<span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"instances"</span><span class="token punctuation">:</span> test_images<span class="token punctuation">.</span>tolist<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token punctuation">)</span><br /><br />resp <span class="token operator">=</span> requests<span class="token punctuation">.</span>post<span class="token punctuation">(</span>url<span class="token punctuation">,</span> data<span class="token operator">=</span>data<span class="token punctuation">,</span> headers<span class="token operator">=</span>headers<span class="token punctuation">)</span><br /><span class="token keyword">if</span> resp<span class="token punctuation">.</span>status_code <span class="token operator">==</span> <span class="token number">200</span><span class="token punctuation">:</span><br /> predictions <span class="token operator">=</span> resp<span class="token punctuation">.</span>json<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token string">'predictions'</span><span class="token punctuation">]</span><br /> res <span class="token operator">=</span> np<span class="token punctuation">.</span>argmax<span class="token punctuation">(</span>predictions<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span><br /> <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"res:"</span><span class="token punctuation">,</span> res<span class="token punctuation">)</span></code></pre>
<p>Not a whole lot of changes from simple prediction.
We pretty much replace the line</p>
<pre class="language-python"><code class="language-python">predictions <span class="token operator">=</span> model<span class="token punctuation">.</span>predict<span class="token punctuation">(</span>test_images<span class="token punctuation">)</span></code></pre>
<p>with the lines</p>
<pre class="language-python"><code class="language-python">resp <span class="token operator">=</span> requests<span class="token punctuation">.</span>post<span class="token punctuation">(</span>url<span class="token punctuation">,</span> data<span class="token operator">=</span>data<span class="token punctuation">,</span> headers<span class="token operator">=</span>headers<span class="token punctuation">)</span><br />predictions <span class="token operator">=</span> resp<span class="token punctuation">.</span>json<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token string">'predictions'</span><span class="token punctuation">]</span></code></pre>
<p>Well, that is pretty much it for running <code>tf-serving</code>.
Now put load balancing on top of it and you got a pretty solid production deployment.</p>
<p>Btw, <a href="https://www.tensorflow.org/tfx/serving/serving_basic">here</a> is <code>tf-serving</code> docs for people who wanna use <code>tensorflow</code>
instead of <code>keras</code> and <code>gRPC</code> instead of <code>REST</code>.</p>
Understanding Kubernetes (Basics)2019-06-26T00:00:00Zhttps://blog.meain.io/2019/understanding-kubernetes/<p>So yeah, you have an amazing app(I am gonna assume it is in python) that you would like to deploy.</p>
<p>You know how to deployment works. You spin up an instance on <code>gcloud</code> or <code>aws</code> or <code>azure</code> or somewhere else.
You install python, no python3, then pip install requirements.txt. Set up your database, maybe redis.
Now you make sure firewall rules are ok if you are deploying on some other port and you start the app and probably even
set up nginx infront because that is what good developers do. Good to go, right?</p>
<p>Well, that is how you would normally do. Now let us see how we do it using Kubernetes.</p>
<blockquote>
<p>Btw, <a href="https://circleci.com/blog/its-the-future/">I'ts the future</a> from circleci a great read.</p>
</blockquote>
<h1 id="basics" tabindex="-1">Basics <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#basics">#</a></h1>
<p>Moving on to how to actually use Kubernetes. I will initially flush out the basic concepts, then go into a simple
explanation of how to deploy your stuff.</p>
<p>You can think of Kubernetes as a collection of different types of wrappers(which has a hierarchy) that you can configure and put on top of
others. The lowest layer is your container which you put in a thing called a pod.</p>
<p>The simplest stack can be:</p>
<ul>
<li>pod</li>
<li>deployment</li>
<li>service</li>
</ul>
<p>A pod is the simplest thingy. It is just your container wrapped in a Kubernetes object(that is what Kubernetes called
each of these wrappers).</p>
<p>Right on top of that you have a <code>Deployment</code>. <code>Deployment</code> is what manages pods(pods not pod).
The idea here is this <code>Deployment</code> object is what takes case of things like checking to see if your pod is alive, restart
it, maintain the number of replicas needed, etc.... You can specify something like I need 2 pods always or I need a minimum of 2
pods but scale up to more if there is a heavy load. This can be really useful if you have seasonal traffic. We will go
into detail as to how to do this later.</p>
<p>Now, on top a <code>Deployment</code> you have a <code>Service</code>. A service is what helps you expose your app to the outside world.
Think of it as putting nginx infront of your app if thas helps. You will most likely be using a <code>Service</code> as load
balancer. You could also be using it just to expose your <code>Deployment</code>. For that we use something called a <code>NodePort</code>.</p>
<h1 id="deploying-your-app-to-a-kubernets-cluster" tabindex="-1">Deploying your app to a <code>Kubernets</code> cluster <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#deploying-your-app-to-a-kubernets-cluster">#</a></h1>
<p>Well, let us get to the fun part and deploy your app to a Kubernetes cluster.</p>
<h2 id="creating-a-kubernetes-cluster-and-setting-up-kubectl" tabindex="-1">Creating a Kubernetes cluster and setting up kubectl <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#creating-a-kubernetes-cluster-and-setting-up-kubectl">#</a></h2>
<p>First thing you will need is <a href="https://kubernetes.io/docs/reference/kubectl/kubectl/"><code>kubectl</code></a> which you will have to <a href="https://kubernetes.io/docs/tasks/tools/install-kubectl/">install</a>.
On macOS with homebrew it is as easy as <code>brew install kubernetes-cli</code>.</p>
<p>Well you are gonna need a Kubernetes cluster to experiment with.
There is this thing called <a href="https://kubernetes.io/docs/tasks/tools/install-minikube/">minikube</a> which you can use
inorder to test out Kubernetes locally. It pretty much sets up a VM in your machine and makes it your Kubernetes cluster
with one node(VM instance). Just run <code>minikube start</code> to start minikube.</p>
<p><img src="https://blog.meain.io/img/minikube.png" alt="screenshot" /></p>
<p>You will need to connect your Kubernetes cluster to <code>kubectl</code>. When you run <code>minikube start</code> it will set up the cluster
and connect it to <code>kubectl</code>. When creating a Kubernetes cluster using gcloud or aws you will have their own way in which
you can connect it to <code>kubectl</code>.</p>
<h1 id="deploying-your-app" tabindex="-1">Deploying your app <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#deploying-your-app">#</a></h1>
<p>I have really simple flask app that needs to be deployed.
<a href="https://github.com/meain/kubernetes-example/tree/master/flaskapp">Here</a> is the app that we have to deploy.</p>
<p>The first step to getting something onto your kubernets cluster is to have a <a href="https://blog.meain.io/2018/docker-basics-react-app/">Dockerfile</a>.
You can see the <a href="https://github.com/meain/kubernetes-example/blob/master/flaskapp/Dockerfile">Dockerfile that I am using</a> in that folder.</p>
<p>You build your docker container, then using a <code>Deployment</code> you set up pods.</p>
<h3 id="build-docker-container" tabindex="-1">Build Docker container <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#build-docker-container">#</a></h3>
<p>Firstly build your Docker container by running the following command in the <code>flaskapp</code> dir.</p>
<pre><code>docker build -t docker.io/meain/flaskapp .
-----
dockerhub username
</code></pre>
<blockquote>
<p>replace <code>meain</code> with your dockerhub username</p>
</blockquote>
<p>This command will build your docker container and tag it with <code>docker.io/meain/flaskapp</code>.
You tag the image with the location where you need to push the image to.</p>
<h3 id="test-your-docker-container" tabindex="-1">Test your docker container <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#test-your-docker-container">#</a></h3>
<p>You can start your docker container by doing <code>docker run --rm -p 8080:8080 docker.io/meain/flaskapp</code>.
This will start your docker container and map the internal 8080 port to the 8080 port in your local machine.</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">docker</span> run <span class="token parameter variable">--rm</span> <span class="token parameter variable">-p</span> <span class="token number">8080</span>:8080 docker.io/meain/flaskapp<br /> * Environment: production<br /> WARNING: This is a development server. Do not use it <span class="token keyword">in</span> a production deployment.<br /> Use a production WSGI server instead.<br /> * Debug mode: off<br /> * Running on http://0.0.0.0:8080/ <span class="token punctuation">(</span>Press CTRL+C to quit<span class="token punctuation">)</span><br /><span class="token number">172.17</span>.0.1 - - <span class="token punctuation">[</span><span class="token number">26</span>/Jun/2019 05:58:36<span class="token punctuation">]</span> <span class="token string">"GET / HTTP/1.1"</span> <span class="token number">200</span> -</code></pre>
<hr />
<pre class="language-shell"><code class="language-shell">$ <span class="token function">curl</span> localhost:8080<br />Hello, World<span class="token operator">!</span>%</code></pre>
<h3 id="pushing-your-image" tabindex="-1">Pushing your image <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#pushing-your-image">#</a></h3>
<p>When deploying, you will have to push your docker image to some remote location and give
that location for image. There are multiple services which let you upload your docker images.
Google has <a href="https://gcr.io/">gcr.io</a> or you could just use <a href="https://hub.docker.com/">hub.docker.com</a>.</p>
<p>You could push your docker image using:</p>
<pre><code>$ push docker.io/meain/flaskapp
</code></pre>
<h3 id="let-us-deploy-already" tabindex="-1">Let us deploy already <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#let-us-deploy-already">#</a></h3>
<p>OK, now that we are tested and ready, let us deploy to Kubernetes.</p>
<p>You can set up a deployment using:</p>
<pre><code>$ kubectl run flaskapp --image='docker.io/meain/flaskapp' --port 8080
--------- ------------------------
deployment-name image-location
</code></pre>
<p>You can choose any deployment name, it does not have to be the same.</p>
<p>Now that you have created a <code>Deployment</code> object, you can query Kubernetes to see that stats.</p>
<pre><code>$ kubectl get pods
NAME READY STATUS RESTARTS AGE
flaskapp-576b787759-4jmd2 1/1 Running 0 3m47s
$ kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
flaskapp 1/1 1 1 3m51s
</code></pre>
<p>Now to set up a <code>Service</code> object and expose the deployment. You can do that using:</p>
<pre><code>$ kubectl expose deployment flaskapp --type=LoadBalancer --port 80 --target-port 8080
--------
service-name
</code></pre>
<p>This will set up a service called <code>flaskapp</code> with type as LoadBalancer. It maps the pod's 8080 port as the port 80 of
the service.</p>
<h3 id="testing-your-deployed-service" tabindex="-1">Testing your deployed service <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#testing-your-deployed-service">#</a></h3>
<p>Now that we have everything deployed, let us test it out.
Well you could just do a <code>curl</code>, but to which address.</p>
<p>If you run</p>
<pre><code>$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
flaskapp LoadBalancer 10.99.64.231 <pending> 80:31852/TCP 3m29s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 59m
</code></pre>
<p>It will list of the services. For services like <code>gke</code> or <code>eks</code>, you will have some value in the <code>EXTERNAL-IP</code> field.
But when using <code>minikube</code> you will only have a <code>CLUSTER-IP</code>.</p>
<p>So, when using with <code>gke</code> or <code>eks</code>, you can just curl to that location.</p>
<p>When using <code>minikube</code>, you create another pod and then use the internal IP.
You can create a tiny pod using the following command. Think of it as <code>ssh</code> ing into a VM instance in the cluster.</p>
<pre><code>$ kubectl run curl --image=radial/busyboxplus:curl -i --tty
</code></pre>
<p>once you are in, you can do a curl the internal ip</p>
<pre><code>$ kubectl run curl --image=radial/busyboxplus:curl -i --tty
If you don't see a command prompt, try pressing enter.
[ root@curl-66bdcf564-4p9lb:/ ]$ curl 10.99.64.231
Hello, World!
</code></pre>
<h3 id="scaling-your-service" tabindex="-1">Scaling your service <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#scaling-your-service">#</a></h3>
<p>You can easily scale a deployment using <code>kubectl</code>. In order to scale our <code>flaskapp</code> deployment to 3 pods we can run.</p>
<pre><code>$ kubectl scale deployment flaskapp --replicas 3
deployment.extensions/flaskapp scaled
</code></pre>
<p>Now, if you check number of pods, you can see 3 pods running.</p>
<pre><code>$ kubectl get pods
NAME READY STATUS RESTARTS AGE
flaskapp-576b787759-4jmd2 1/1 Running 0 39m
flaskapp-576b787759-8bhdc 1/1 Running 0 84s
flaskapp-576b787759-kdps9 1/1 Running 0 84s
...
</code></pre>
<p>Awesome, right?</p>
<p>OK, now autoscaling. It is just as easy. You can run:</p>
<pre><code>$ kubectl autoscale deployment flaskapp --min=1 --max=5 --cpu-percent=50
horizontalpodautoscaler.autoscaling/flaskapp autoscaled
</code></pre>
<p>Now with that, it will scale up from 1 pod to a max of 5 pods based on the <code>cpu-percent</code>. [no of nodes = ceil(cpu-percen/50)]</p>
<h3 id="deleting-stuff" tabindex="-1">Deleting stuff <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#deleting-stuff">#</a></h3>
<p>Well, it is really easy to delete your deployment or service. You can run:</p>
<pre><code>kubectl delete pod <pod-name>
kubectl delete deployment <deployment-name>
kubectl delete service <service-name>
</code></pre>
<p>Well, that is pretty much the basics.</p>
<h1 id="writing-config-files" tabindex="-1">Writing config files <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#writing-config-files">#</a></h1>
<p>Well, all this is pretty cool. But, you probably don't wanna retype the entire command with all the parameters every
time you wanna deploy something or make some changes. That is exactly why you have a config file.</p>
<p>You can have a config file for any kind of kubernetes object. It is a yaml file which will essentially specify the
parameters which you would be specifying in the command that you would enter.</p>
<blockquote>
<p>You could have config for multiple kubernets object in one file, you will just have to seperate it with <code>---</code>.</p>
</blockquote>
<p>I am not going into all the configuration options that you can do because it is pretty vast, and it is bettter that
you just go through the <a href="https://kubernetes.io/">documentation for kubernetes</a>.</p>
<p>I will go through the config for a deployment and a service.</p>
<h3 id="deployment-(sample-config)" tabindex="-1">Deployment (Sample config) <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#deployment-(sample-config)">#</a></h3>
<p>To do the same deployment as what the command above, we will be writing a config file which looks something like
this:</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
name: flask
spec:
selector:
matchLabels:
app: flask replicas: 1
template:
metadata:
labels:
app: flask
spec:
containers:
- name: flask
image: "docker.io/meain/flaskapp"
ports:
- name: backend
containerPort: 8080
</code></pre>
<p>Let me introduce you to the important pieces.
Almost every kubernetes config piece will contain these things:</p>
<ul>
<li><code>apiVersion</code>: the version of the kubernets config</li>
<li><code>kind</code>: kind of object that you are working with</li>
<li><code>metadata</code>: things like <code>name</code>, <code>namespace</code>, <code>labels</code>, etc...</li>
<li><code>spec</code>: the spec of the deployment, as in which pod, how many replicas etc..</li>
</ul>
<blockquote>
<p>You use labels or name to link between different objects</p>
</blockquote>
<p>The <code>template</code> section of a deployment could be actually defined in a <code>Pod</code> object and connected via labels.
In this case, if we were to separate out the two, we could have the <code>template</code> section in a different file and have this <code>Deployment</code> config linked to it
by using <code>selector > matchLabels</code> in the <code>Deployment</code> section and matching it to <code>metadata > labels</code> in the <code>Pod</code> section of that config.</p>
<p>But in this case we just specify the <code>Pod</code> specification directly in the <code>Deployment</code> piece.</p>
<p>In here the <code>spec</code> for <code>Deployment</code> specifies:</p>
<ul>
<li><code>selector</code>: specify labels used to select the <code>Pod</code></li>
<li><code>replicas</code>: how many replicas of the pod to use</li>
<li><code>template</code>: this specified the pod definition (could have been taken out in to a separate piece if needed)</li>
</ul>
<p>The <code>template</code> section actually contains the <code>Pod</code> definition.
It will have your <code>metadata</code> section, but don't need others like <code>apiVersion</code>, <code>kind</code> etc as they are already know.
<code>apiVersion</code> as it is specified for <code>Deployment</code> and <code>kind</code> can be inferred from the fact that this a <code>template</code> in
<code>Deployment</code> config.</p>
<p>So the <code>Pod</code> spec contains:</p>
<ul>
<li><code>container</code>: the pod specification
<ul>
<li><code>name</code>: name of the pod</li>
<li><code>image</code>: docker image link</li>
<li><code>ports</code>: ports to be exposed</li>
</ul>
</li>
</ul>
<p>Well that is pretty much it for a <code>Deployment</code> config.
Now here is a sample <code>Service</code> config.</p>
<h3 id="service-(sample-config)" tabindex="-1">Service (Sample config) <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#service-(sample-config)">#</a></h3>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> Service<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> flask<br /> <span class="token key atrule">labels</span><span class="token punctuation">:</span><br /> <span class="token key atrule">app</span><span class="token punctuation">:</span> flask<br /> <span class="token key atrule">tier</span><span class="token punctuation">:</span> backend<br /> <span class="token key atrule">track</span><span class="token punctuation">:</span> stable<br /><span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">selector</span><span class="token punctuation">:</span><br /> <span class="token key atrule">app</span><span class="token punctuation">:</span> flask<br /> <span class="token key atrule">tier</span><span class="token punctuation">:</span> backend<br /> <span class="token key atrule">ports</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">protocol</span><span class="token punctuation">:</span> TCP<br /> <span class="token key atrule">port</span><span class="token punctuation">:</span> <span class="token number">80</span><br /> <span class="token key atrule">targetPort</span><span class="token punctuation">:</span> backend<br /> <span class="token key atrule">type</span><span class="token punctuation">:</span> LoadBalancer</code></pre>
<p>Here, again you have the same labels like <code>apiVersion</code>, <code>kind</code>, <code>metadata</code> etc...
But the <code>spec</code> section changes.</p>
<p>For the <code>spec</code> of a service we define things like:</p>
<ul>
<li><code>ports</code>: ports to be connected to</li>
<li><code>type</code>: type of service</li>
</ul>
<p>There is a lot more things that the <code>spec</code> section of <code>Deployment</code> or <code>Seervice</code> can handle. You can look them up in the
documentation for kubernetes. But this is the essential idea.</p>
<h2 id="using-config-files" tabindex="-1">Using config files <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#using-config-files">#</a></h2>
<p>OK, now you have the config files. How to use them?</p>
<p>You can either run:</p>
<p><code>kubectl apply -f <config-file></code> or <code>kubectl create -f <config-file></code></p>
<p>The only difference is that if you use <code>create</code>, you will not be able to change the config later.
With <code>apply</code>, you could change the file later and run the same command again to apply only the changes that you have
made.</p>
<h1 id="other-resources" tabindex="-1">Other resources <a class="direct-link" href="https://blog.meain.io/2019/understanding-kubernetes/#other-resources">#</a></h1>
<ul>
<li><a href="https://kubernetes.io/">Kubernetes Documentation</a></li>
<li><a href="https://cloud.google.com/kubernetes-engine/docs/tutorials/hello-app">Deploying a containerized web application</a></li>
<li><a href="https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes">How to Set Up an Nginx Ingress with Cert-Manager on DigitalOcean Kubernetes</a></li>
</ul>
<p>With that, I'm out. You are on your own now. But I guess that it helped out a bit.</p>
Setting local files as New Tab page in Firefox2019-07-27T00:00:00Zhttps://blog.meain.io/2019/set-local-file-as-newtabpage-in-firefox/<blockquote>
<p>This will not work as of Firefox 72</p>
</blockquote>
<p>So, I was going through <a href="https://www.reddit.com/r/startpages/"><code>r/startpages</code></a> and checking out the <code>startpages</code> there.
A lot of them looked pretty awesome and I decided to create one. And create, I did <a href="https://github.com/meain/startpage">meain/startpage</a>.</p>
<p>But yea, setting a local file as startpage is a messy deal in Firefox.
Lot of people seem to set up a local server and even after than end up having to use a plugin to get things working.</p>
<p>Here is a bit complex but much better way to do it. Lets get started.</p>
<p>Firefox lets you change the css of the browser by providing custom css file called <a href="https://github.com/meain/dotfiles/blob/master/firefox/userChrome.css">userChrome.css</a>.
What we do is, load a javascript file using this css file and change Firefox setting using that js file.</p>
<blockquote>
<p>As of Firefox 69, you might have to manually set the value of <code>toolkit.legacyUserProfileCustomizations.stylesheets</code> to
<code>true</code> in <code>about:config</code></p>
</blockquote>
<h3 id="set-up-userchrome.css" tabindex="-1">Set up <code>userChrome.css</code> <a class="direct-link" href="https://blog.meain.io/2019/set-local-file-as-newtabpage-in-firefox/#set-up-userchrome.css">#</a></h3>
<p>The first step is to create a file called <code>userChrome.css</code> file in your <code><profile>/chrome</code> directory.
To find out Firefox's profile directory, you can check in <code>Help</code> > <code>Troubleshooting Information</code> section in the
hamburger menu in the top right of the browser.</p>
<p>You will land in a page which look something like this.</p>
<p><img src="https://i.imgur.com/VpCfMAG.png" alt="Troubleshooting Info Page" /></p>
<p>Mine is <code>/Users/meain/Library/Application Support/Firefox/Profiles/jxqrburh.default-release</code>.</p>
<p>So you go to this folder. There may or may not be a folder called <code>chrome</code>. In any case create a file named
<code>userChrome.css</code> in the <code>chrome</code> folder inside your profile folder.</p>
<h3 id="use-userchrome.css-to-load-js-file" tabindex="-1">Use <code>userChrome.css</code> to load js file <a class="direct-link" href="https://blog.meain.io/2019/set-local-file-as-newtabpage-in-firefox/#use-userchrome.css-to-load-js-file">#</a></h3>
<p>Now let us get the css file to load the js file, in our case <code>userChrome.js</code>.
Credis to <a href="https://github.com/Sporif/firefox-quantum-userchromejs">Sporif/firefox-quantum-userchromejs</a>.</p>
<p>Add this code to <code>userChrome.css</code> file:</p>
<pre class="language-css"><code class="language-css"><span class="token atrule"><span class="token rule">@namespace</span> <span class="token url"><span class="token function">url</span><span class="token punctuation">(</span>http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul<span class="token punctuation">)</span></span><span class="token punctuation">;</span></span><br /><br /><span class="token selector">toolbarbutton#alltabs-button</span> <span class="token punctuation">{</span><br /> <span class="token property">-moz-binding</span><span class="token punctuation">:</span> <span class="token url"><span class="token function">url</span><span class="token punctuation">(</span><span class="token string url">"userChrome.xml#js"</span><span class="token punctuation">)</span></span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>You will also need to add two other files. <code>userChrome.xml</code> and <code>userChrome.js</code>.
We will go over what needs to be done in <code>userChrome.js</code> but for <code>userChrome.xml</code> add the below content.</p>
<pre class="language-xml"><code class="language-xml"><span class="token prolog"><?xml version="1.0"?></span><br /><span class="token comment"><!-- Copyright (c) 2017 Haggai Nuchi<br />Available for use under the MIT License:<br />https://opensource.org/licenses/MIT<br /> --></span><br /><br /><span class="token comment"><!-- Run userChrome.js/userChrome.xul and .uc.js/.uc.xul/.css files --></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>bindings</span> <span class="token attr-name">xmlns</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>http://www.mozilla.org/xbl<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>binding</span> <span class="token attr-name">id</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>js<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>implementation</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>constructor</span><span class="token punctuation">></span></span><span class="token cdata"><![CDATA[<br /> if(window.userChromeJsMod) return;<br /> window.userChromeJsMod = true;<br /> var chromeFiles = FileUtils.getDir("UChrm", []).directoryEntries;<br /> var xulFiles = [];<br /> var sss = Cc['@mozilla.org/content/style-sheet-service;1'].getService(Ci.nsIStyleSheetService);<br /> while(chromeFiles.hasMoreElements()) {<br /> var file = chromeFiles.getNext().QueryInterface(Ci.nsIFile);<br /> var fileURI = Services.io.newFileURI(file);<br /> if(file.isFile()) {<br /> type = "none";<br /> if(/(^userChrome|\.uc)\.js$/i.test(file.leafName)) {<br /> type = "userchrome/js";<br /> }<br /> else if(/(^userChrome|\.uc)\.xul$/i.test(file.leafName)) {<br /> type = "userchrome/xul";<br /> }<br /> else if(/\.as\.css$/i.test(file.leafName)) {<br /> type = "agentsheet";<br /> }<br /> else if(/^(?!(userChrome|userContent)\.css$).+\.css$/i.test(file.leafName)) {<br /> type = "usersheet";<br /> }<br /> if(type != "none") {<br /> console.log("----------\\ " + file.leafName + " (" + type + ")");<br /> try {<br /> if(type == "userchrome/js") {<br /> Services.scriptloader.loadSubScriptWithOptions(fileURI.spec, {target: window, ignoreCache: true});<br /> }<br /> else if(type == "userchrome/xul") {<br /> xulFiles.push(fileURI.spec);<br /> }<br /> else if(type == "agentsheet") {<br /> if(!sss.sheetRegistered(fileURI, sss.AGENT_SHEET))<br /> sss.loadAndRegisterSheet(fileURI, sss.AGENT_SHEET);<br /> }<br /> else if(type == "usersheet") {<br /> if(!sss.sheetRegistered(fileURI, sss.USER_SHEET))<br /> sss.loadAndRegisterSheet(fileURI, sss.USER_SHEET);<br /> }<br /> } catch(e) {<br /> console.log("########## ERROR: " + e + " at " + e.lineNumber + ":" + e.columnNumber);<br /> }<br /> console.log("----------/ " + file.leafName);<br /> }<br /> }<br /> }<br /> setTimeout(function loadXUL() {<br /> if(xulFiles.length > 0) {<br /> document.loadOverlay(xulFiles.shift(), null);<br /> setTimeout(loadXUL, 5);<br /> }<br /> }, 0);<br /> ]]></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>constructor</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>implementation</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>binding</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>bindings</span><span class="token punctuation">></span></span></code></pre>
<h3 id="setting-up-userchrome.js" tabindex="-1">Setting up <code>userChrome.js</code> <a class="direct-link" href="https://blog.meain.io/2019/set-local-file-as-newtabpage-in-firefox/#setting-up-userchrome.js">#</a></h3>
<p>Now finally we have to add in the code to change Firefox settings.</p>
<p>Add the below code in <code>userChrome.js</code> but replace the value of <code>mypage</code> to the location of your <code>index.html</code> file.</p>
<pre class="language-js"><code class="language-js"><span class="token punctuation">(</span><span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token comment">// IMPORTANT: when there's no filename, be sure to include a trailing slash at the end.</span><br /> <span class="token keyword">const</span> mypage <span class="token operator">=</span> <span class="token string">"file:///Users/meain/Documents/Projects/projects/startpage/index.html"</span><span class="token punctuation">;</span><br /> <span class="token comment">// Don't place the caret in the location bar. Useful if you want a page's search box to have focus instead.</span><br /> <span class="token keyword">var</span> removefocus <span class="token operator">=</span> <span class="token string">"no"</span><span class="token punctuation">;</span><br /> <span class="token comment">// Clear the page's URL from the location bar. Normally not needed, as this should already be the default behavior.</span><br /> <span class="token keyword">var</span> clearlocationbar <span class="token operator">=</span> <span class="token string">"no"</span><span class="token punctuation">;</span><br /><br /> aboutNewTabService<span class="token punctuation">.</span>newTabURL <span class="token operator">=</span> mypage<span class="token punctuation">;</span><br /> <span class="token keyword">function</span> <span class="token function">customNewTab</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>removefocus <span class="token operator">==</span> <span class="token string">"yes"</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">setTimeout</span><span class="token punctuation">(</span><span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> gBrowser<span class="token punctuation">.</span>selectedBrowser<span class="token punctuation">.</span><span class="token function">focus</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>clearlocationbar <span class="token operator">==</span> <span class="token string">"yes"</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">setTimeout</span><span class="token punctuation">(</span><span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>gBrowser<span class="token punctuation">.</span>selectedBrowser<span class="token punctuation">.</span>currentURI<span class="token punctuation">.</span>spec <span class="token operator">==</span> mypage<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> window<span class="token punctuation">.</span>document<span class="token punctuation">.</span><span class="token function">getElementById</span><span class="token punctuation">(</span><span class="token string">"urlbar"</span><span class="token punctuation">)</span><span class="token punctuation">.</span>value <span class="token operator">=</span> <span class="token string">""</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">1000</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> gBrowser<span class="token punctuation">.</span>tabContainer<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">"TabOpen"</span><span class="token punctuation">,</span> customNewTab<span class="token punctuation">,</span> <span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /><span class="token punctuation">}</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>Restart Firefox and you should be good to go.
If you need a good startpage, feel free to check out mine at <a href="https://github.com/meain/startpage">meain/startpage</a>.</p>
<h3 id="extra" tabindex="-1">Extra <a class="direct-link" href="https://blog.meain.io/2019/set-local-file-as-newtabpage-in-firefox/#extra">#</a></h3>
<p>Well, you could use <code>userChrome.css</code> and <code>userChrome.js</code> for more than just this.
Here are a few websites you might wanna check out if you would like to know more what they can do.</p>
<ul>
<li><a href="https://www.userchrome.org/">userchrome.org</a></li>
<li><a href="https://luke-baker.github.io/">luke-baker.github.io</a></li>
<li><a href="https://github.com/alice0775/userChrome.js/">alice0775/userChrome.js</a></li>
</ul>
Git never forgets, well kinda. `git-reflog`2019-08-10T00:00:00Zhttps://blog.meain.io/2019/git_never_forgets_well_kinda/<p>Hi, I am gonna let you in on a little secret.
Git never forgets what you do, you can mess up however you want and you will still have all your data available*.</p>
<blockquote>
<p>* as long as you have committed you code and also only for 90 days</p>
</blockquote>
<p>OK, I am not taking about the fact that you can you can see the logs of what changes you have made over a period of
time. I am talking about that fact that you can even recover from something like <code>git reset --hard HEAD~30</code>.</p>
<p>Working with git is a little scary for a lot of people. Things like <code>rebasing</code>, <code>resetting</code> etc.
I know a lot of people who only ever does <code>git push</code>, <code>git commit</code>, and <code>git pull</code>.</p>
<p><img src="https://imgs.xkcd.com/comics/git.png" alt="xkcd" /></p>
<p>I am here to say that you are fine, you can do any kind of weird shit and still have everything there.
Let me introduce you to the magical command:</p>
<pre><code>git reflog
</code></pre>
<p>Here is what a sample output of <code>git reflog</code> will look like:</p>
<p><img src="https://blog.meain.io/img/gitreflog.png" alt="screenshot" /></p>
<p>This is a rather busy tree. As you can see, every check out, every pull, every reset. All of them are in there.</p>
<h3 id="ok%2C-so-how-do-i-use-them%3F" tabindex="-1">OK, so how do I use them? <a class="direct-link" href="https://blog.meain.io/2019/git_never_forgets_well_kinda/#ok%2C-so-how-do-i-use-them%3F">#</a></h3>
<p>It is simple, let us consider a simple mistake that you did. Let's say that you accidentally hard reset to <code>HEAD~30</code>.</p>
<p><em>If you don't know what that does, It resets your git commit to 30 commits behind current commit.
This is different form checking out HEAD~30 as in this case it rewrites the tree.</em></p>
<pre><code>git reset --hard HEAD~30
</code></pre>
<p>OK, now what do you do. Delete the folder and clone again is a possibility. But let's do it the right way.</p>
<p><strong>Run <code>git reflog</code>. Mine looks something like this:</strong></p>
<pre><code>dfd1d3e (HEAD -> master) HEAD@{0}: reset: moving to HEAD~30
a4d53b0 (origin/master) HEAD@{1}: commit: post:rust-macros: add output and code link
6cc6335 HEAD@{2}: commit: post:local-file-as-newtab-firefox
b597ae7 HEAD@{3}: commit: config: fix rss link generated
afc0283 HEAD@{4}: commit: post:kubernetes
7bd3eee HEAD@{5}: commit: post:tf-serving:add tf-serving docs link
77462fb HEAD@{6}: commit (amend): post:tf-serving
07fdc2e HEAD@{7}: commit: post:tf-serving
</code></pre>
<p>You can see that we have everything that I have done in the past listed here.
Also if you check the latest one, it does show that I did reset it to <code>HEAD~30</code>.</p>
<p>Now to restore the original state, we just do a hard reset to the one just before I did reset to <code>HEAD~30</code>.
In my case it will be to <code>a4d53b0</code>.</p>
<p>So I will run:</p>
<pre><code>git reset --hard a4d53b0
</code></pre>
<p>After running this we are back to where we started.
Now, if I were to run <code>git reflog</code>, I would get something like:</p>
<pre><code>a4d53b0 (HEAD -> master, origin/master) HEAD@{0}: reset: moving to a4d53b0
dfd1d3e HEAD@{1}: reset: moving to HEAD~30
a4d53b0 (HEAD -> master, origin/master) HEAD@{2}: commit: post:rust-macros: add output and code link
6cc6335 HEAD@{3}: commit: post:local-file-as-newtab-firefox
b597ae7 HEAD@{4}: commit: config: fix rss link generated
afc0283 HEAD@{5}: commit: post:kubernetes
7bd3eee HEAD@{6}: commit: post:tf-serving:add tf-serving docs link
77462fb HEAD@{7}: commit (amend): post:tf-serving
</code></pre>
<p>t is not only applicable for just resets, whatever you do, you can come here and revert it.
I hope this make git a little less scary.</p>
Accessing Kubernetes API from a Pod (RBAC)2019-08-12T00:00:00Zhttps://blog.meain.io/2019/accessing-kubernetes-api-from-pod/<p>This article describes in general how to set up permission for a Pod so that it will have access to Kubernetes API.</p>
<p>My exact use case was that, I wanted to run a Pod which will watch a redis queue and then start a job whenever there is
a new item in the queue. I will only be explaining how to set up the permissions and I think the rest of the tasks has
been well explained by a lot of people.</p>
<p><img src="https://blog.meain.io/img/job.gif" alt="gif" /></p>
<p>So, I have a simple python file that I would like to run to create a <code>Job</code> and then delete it later.
It will look something like this:</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">from</span> kubernetes <span class="token keyword">import</span> client<span class="token punctuation">,</span> config<br /><br />JOB_NAME <span class="token operator">=</span> <span class="token string">"pi"</span><br /><br /><br /><span class="token keyword">def</span> <span class="token function">create_job_object</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token comment"># Configureate Pod template container</span><br /> container <span class="token operator">=</span> client<span class="token punctuation">.</span>V1Container<span class="token punctuation">(</span><br /> name<span class="token operator">=</span><span class="token string">"pi"</span><span class="token punctuation">,</span><br /> image<span class="token operator">=</span><span class="token string">"perl"</span><span class="token punctuation">,</span><br /> command<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"perl"</span><span class="token punctuation">,</span> <span class="token string">"-Mbignum=bpi"</span><span class="token punctuation">,</span> <span class="token string">"-wle"</span><span class="token punctuation">,</span> <span class="token string">"print bpi(2000)"</span><span class="token punctuation">]</span><span class="token punctuation">)</span><br /> <span class="token comment"># Create and configurate a spec section</span><br /> template <span class="token operator">=</span> client<span class="token punctuation">.</span>V1PodTemplateSpec<span class="token punctuation">(</span><br /> metadata<span class="token operator">=</span>client<span class="token punctuation">.</span>V1ObjectMeta<span class="token punctuation">(</span>labels<span class="token operator">=</span><span class="token punctuation">{</span><span class="token string">"app"</span><span class="token punctuation">:</span> <span class="token string">"pi"</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> spec<span class="token operator">=</span>client<span class="token punctuation">.</span>V1PodSpec<span class="token punctuation">(</span>restart_policy<span class="token operator">=</span><span class="token string">"Never"</span><span class="token punctuation">,</span> containers<span class="token operator">=</span><span class="token punctuation">[</span>container<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token comment"># Create the specification of deployment</span><br /> spec <span class="token operator">=</span> client<span class="token punctuation">.</span>V1JobSpec<span class="token punctuation">(</span><br /> template<span class="token operator">=</span>template<span class="token punctuation">,</span><br /> backoff_limit<span class="token operator">=</span><span class="token number">4</span><span class="token punctuation">)</span><br /> <span class="token comment"># Instantiate the job object</span><br /> job <span class="token operator">=</span> client<span class="token punctuation">.</span>V1Job<span class="token punctuation">(</span><br /> api_version<span class="token operator">=</span><span class="token string">"batch/v1"</span><span class="token punctuation">,</span><br /> kind<span class="token operator">=</span><span class="token string">"Job"</span><span class="token punctuation">,</span><br /> metadata<span class="token operator">=</span>client<span class="token punctuation">.</span>V1ObjectMeta<span class="token punctuation">(</span>name<span class="token operator">=</span>JOB_NAME<span class="token punctuation">)</span><span class="token punctuation">,</span><br /> spec<span class="token operator">=</span>spec<span class="token punctuation">)</span><br /><br /> <span class="token keyword">return</span> job<br /><br /><br /><span class="token keyword">def</span> <span class="token function">create_job</span><span class="token punctuation">(</span>api_instance<span class="token punctuation">,</span> job<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token comment"># Create job</span><br /> api_response <span class="token operator">=</span> api_instance<span class="token punctuation">.</span>create_namespaced_job<span class="token punctuation">(</span><br /> body<span class="token operator">=</span>job<span class="token punctuation">,</span><br /> namespace<span class="token operator">=</span><span class="token string">"default"</span><span class="token punctuation">)</span><br /> <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"Job created. status='%s'"</span> <span class="token operator">%</span> <span class="token builtin">str</span><span class="token punctuation">(</span>api_response<span class="token punctuation">.</span>status<span class="token punctuation">)</span><span class="token punctuation">)</span><br /><br /><br /><span class="token keyword">def</span> <span class="token function">update_job</span><span class="token punctuation">(</span>api_instance<span class="token punctuation">,</span> job<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token comment"># Update container image</span><br /> job<span class="token punctuation">.</span>spec<span class="token punctuation">.</span>template<span class="token punctuation">.</span>spec<span class="token punctuation">.</span>containers<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>image <span class="token operator">=</span> <span class="token string">"perl"</span><br /> <span class="token comment"># Update the job</span><br /> api_response <span class="token operator">=</span> api_instance<span class="token punctuation">.</span>patch_namespaced_job<span class="token punctuation">(</span><br /> name<span class="token operator">=</span>JOB_NAME<span class="token punctuation">,</span><br /> namespace<span class="token operator">=</span><span class="token string">"default"</span><span class="token punctuation">,</span><br /> body<span class="token operator">=</span>job<span class="token punctuation">)</span><br /> <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"Job updated. status='%s'"</span> <span class="token operator">%</span> <span class="token builtin">str</span><span class="token punctuation">(</span>api_response<span class="token punctuation">.</span>status<span class="token punctuation">)</span><span class="token punctuation">)</span><br /><br /><br /><span class="token keyword">def</span> <span class="token function">delete_job</span><span class="token punctuation">(</span>api_instance<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token comment"># Delete job</span><br /> api_response <span class="token operator">=</span> api_instance<span class="token punctuation">.</span>delete_namespaced_job<span class="token punctuation">(</span><br /> name<span class="token operator">=</span>JOB_NAME<span class="token punctuation">,</span><br /> namespace<span class="token operator">=</span><span class="token string">"default"</span><span class="token punctuation">,</span><br /> body<span class="token operator">=</span>client<span class="token punctuation">.</span>V1DeleteOptions<span class="token punctuation">(</span><br /> propagation_policy<span class="token operator">=</span><span class="token string">'Foreground'</span><span class="token punctuation">,</span><br /> grace_period_seconds<span class="token operator">=</span><span class="token number">5</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"Job deleted. status='%s'"</span> <span class="token operator">%</span> <span class="token builtin">str</span><span class="token punctuation">(</span>api_response<span class="token punctuation">.</span>status<span class="token punctuation">)</span><span class="token punctuation">)</span><br /><br /><br /><span class="token keyword">def</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span><br /> <span class="token comment"># Configs can be set in Configuration class directly or using helper</span><br /> <span class="token comment"># utility. If no argument provided, the config will be loaded from</span><br /> <span class="token comment"># default location.</span><br /><br /> <span class="token comment"># config.load_kube_config()</span><br /> config<span class="token punctuation">.</span>load_incluster_config<span class="token punctuation">(</span><span class="token punctuation">)</span><br /> batch_v1 <span class="token operator">=</span> client<span class="token punctuation">.</span>BatchV1Api<span class="token punctuation">(</span><span class="token punctuation">)</span><br /><br /> <span class="token comment"># Create a job object with client-python API. The job we</span><br /> job <span class="token operator">=</span> create_job_object<span class="token punctuation">(</span><span class="token punctuation">)</span><br /><br /> create_job<span class="token punctuation">(</span>batch_v1<span class="token punctuation">,</span> job<span class="token punctuation">)</span><br /> jobs <span class="token operator">=</span> batch_v1<span class="token punctuation">.</span>list_namespaced_job<span class="token punctuation">(</span><span class="token string">'default'</span><span class="token punctuation">)</span><br /> jobs<span class="token punctuation">.</span>items<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>status<br /><br /> update_job<span class="token punctuation">(</span>batch_v1<span class="token punctuation">,</span> job<span class="token punctuation">)</span><br /><br /> delete_job<span class="token punctuation">(</span>batch_v1<span class="token punctuation">)</span><br /><br /><br /><span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">'__main__'</span><span class="token punctuation">:</span><br /> main<span class="token punctuation">(</span><span class="token punctuation">)</span></code></pre>
<p>Here is the gist of the architecture.
We have a <code>Pod</code> running in the <code>default</code> namespace.
We will be creating and later deleting a <code>Job</code> from within this namespace.</p>
<p>So, what we will have to do is to allow the <code>Pod</code> access to the Kubernetes API so that it can do the various tasks.
If you were to run without giving the needed permissions you will end up getting a error message like:</p>
<pre><code>kubernetes.client.rest.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Audit-Id': 'e518f584-364d-40b7-a6d1-d4528062298d', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'Date': 'Mon, 12 Aug 2019 08:11:29 GMT', 'Content-Length': '313'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"jobs.batch is forbidden: User \"system:serviceaccount:default:job-robot\" cannot create resource \"jobs\" in API group \"batch\" in the namespace \"default\"","reason":"Forbidden","details":{"group":"batch","kind":"jobs"},"code":403}
</code></pre>
<h2 id="architecture" tabindex="-1">Architecture <a class="direct-link" href="https://blog.meain.io/2019/accessing-kubernetes-api-from-pod/#architecture">#</a></h2>
<p><img src="https://blog.meain.io/img/kube-rbac.png" alt="RBAC visualization" /></p>
<ul>
<li>
<p>Create a new <code>ServiceAccount</code> object.
We will later create a<code>Pod</code> which we assign to this <code>ServiceAccount</code></p>
</li>
<li>
<p>Create a new <code>Role</code>. A <code>Role</code> is instructions on what a <code>ServiceAccount</code> will have access to.
It will not say which <code>ServiceAccount</code> will have access.</p>
</li>
<li>
<p>We can say that specific <code>ServiceAccount</code> will have the <code>Role</code> rules by using a <code>RoleBinding</code>.</p>
</li>
<li>
<p>Now you create a <code>Pod</code> which will be assigned to the created <code>ServiceAccount</code>.</p>
</li>
</ul>
<blockquote>
<p>The difference between <code>Role{,Binding}</code> and <code>ClusterRole{,Binding}</code> is that in the latter, you apply it for all
namespaces.</p>
</blockquote>
<h2 id="code" tabindex="-1">Code <a class="direct-link" href="https://blog.meain.io/2019/accessing-kubernetes-api-from-pod/#code">#</a></h2>
<p>Below is some sample yaml file that you could use for reference.</p>
<h4 id="serviceaccount" tabindex="-1">ServiceAccount <a class="direct-link" href="https://blog.meain.io/2019/accessing-kubernetes-api-from-pod/#serviceaccount">#</a></h4>
<p>This can be technically considered like a group(don't confuse with the <code>Group</code> concept), and <code>things</code> belonging to this group can be assigned specific rules.
Here we create a <code>ServiceAccount</code> with the name <code>job-robot</code>.</p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> ServiceAccount<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> job<span class="token punctuation">-</span>robot</code></pre>
<h4 id="role" tabindex="-1">Role <a class="direct-link" href="https://blog.meain.io/2019/accessing-kubernetes-api-from-pod/#role">#</a></h4>
<p>Here we create a <code>Role</code> with the name <code>job-robot</code> (does not have to the same as the service-account).
We let the <code>Role</code> to have access to get,list and watch pods.
Also to get,list,watch,create,update,patch and delete jobs</p>
<p>We will later assign this to the <code>ServiceAccount</code>.</p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> rbac.authorization.k8s.io/v1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> Role<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">namespace</span><span class="token punctuation">:</span> default<br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> job<span class="token punctuation">-</span>robot<br /><span class="token key atrule">rules</span><span class="token punctuation">:</span><br /><span class="token punctuation">-</span> <span class="token key atrule">apiGroups</span><span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token string">""</span><span class="token punctuation">]</span> <span class="token comment"># "" indicates the core API group</span><br /> <span class="token key atrule">resources</span><span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token string">"pods"</span><span class="token punctuation">]</span><br /> <span class="token key atrule">verbs</span><span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token string">"get"</span><span class="token punctuation">,</span> <span class="token string">"list"</span><span class="token punctuation">,</span> <span class="token string">"watch"</span><span class="token punctuation">]</span><br /><span class="token punctuation">-</span> <span class="token key atrule">apiGroups</span><span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token string">"batch"</span><span class="token punctuation">,</span> <span class="token string">"extensions"</span><span class="token punctuation">]</span><br /> <span class="token key atrule">resources</span><span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token string">"jobs"</span><span class="token punctuation">]</span><br /> <span class="token key atrule">verbs</span><span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token string">"get"</span><span class="token punctuation">,</span> <span class="token string">"list"</span><span class="token punctuation">,</span> <span class="token string">"watch"</span><span class="token punctuation">,</span> <span class="token string">"create"</span><span class="token punctuation">,</span> <span class="token string">"update"</span><span class="token punctuation">,</span> <span class="token string">"patch"</span><span class="token punctuation">,</span> <span class="token string">"delete"</span><span class="token punctuation">]</span></code></pre>
<h4 id="rolebinding" tabindex="-1">RoleBinding <a class="direct-link" href="https://blog.meain.io/2019/accessing-kubernetes-api-from-pod/#rolebinding">#</a></h4>
<p>Now we have to bind the <code>Role</code> to that <code>ServiceAccount</code>.
Here we create a <code>RoleBinding</code> with the name <code>job-robot</code> (again, does not have to be the same).
Doing this will bind your <code>Role</code> to the <code>ServiceAccount</code>.</p>
<p>You define you <code>Role</code> in the <code>roleRef</code> section and your <code>ServiceAccount</code> in the <code>subjects</code> section.</p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> rbac.authorization.k8s.io/v1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> RoleBinding<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> job<span class="token punctuation">-</span>robot<br /> <span class="token key atrule">namespace</span><span class="token punctuation">:</span> default<br /><span class="token key atrule">subjects</span><span class="token punctuation">:</span><br /><span class="token punctuation">-</span> <span class="token key atrule">kind</span><span class="token punctuation">:</span> ServiceAccount<br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> job<span class="token punctuation">-</span>robot <span class="token comment"># Name of the ServiceAccount</span><br /> <span class="token key atrule">namespace</span><span class="token punctuation">:</span> default<br /><span class="token key atrule">roleRef</span><span class="token punctuation">:</span><br /> <span class="token key atrule">kind</span><span class="token punctuation">:</span> Role <span class="token comment"># This must be Role or ClusterRole</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> job<span class="token punctuation">-</span>robot <span class="token comment"># This must match the name of the Role or ClusterRole you wish to bind to</span><br /> <span class="token key atrule">apiGroup</span><span class="token punctuation">:</span> rbac.authorization.k8s.io</code></pre>
<h4 id="deployment%2Fpod" tabindex="-1">Deployment/Pod <a class="direct-link" href="https://blog.meain.io/2019/accessing-kubernetes-api-from-pod/#deployment%2Fpod">#</a></h4>
<p>Well, the final step is to put your deployment on to your cluster.
There is one thing that you will have to do.</p>
<p>In your <code>Pod</code> spec, you will have have to add an extra key <code>serviceAccountName</code> with the name of the <code>ServiceAccount</code> you
created.</p>
<pre class="language-yaml"><code class="language-yaml"><br /><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> apps/v1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> Deployment<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> ubu<br /><span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">selector</span><span class="token punctuation">:</span><br /> <span class="token key atrule">matchLabels</span><span class="token punctuation">:</span><br /> <span class="token key atrule">app</span><span class="token punctuation">:</span> ubu<br /> <span class="token key atrule">replicas</span><span class="token punctuation">:</span> <span class="token number">1</span><br /> <span class="token key atrule">template</span><span class="token punctuation">:</span><br /> <span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">labels</span><span class="token punctuation">:</span><br /> <span class="token key atrule">app</span><span class="token punctuation">:</span> ubu<br /> <span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">containers</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> ubu<br /> <span class="token key atrule">image</span><span class="token punctuation">:</span> <span class="token string">"<image-name>"</span><br /> <span class="token key atrule">serviceAccountName</span><span class="token punctuation">:</span> job<span class="token punctuation">-</span>robot <span class="token comment"># Name of the ServiceAccount, duh.</span></code></pre>
<p>And with that, you can now create a <code>Job</code> from within a <code>Pod</code> in your cluster.</p>
Making sure you won't commit conflict markers2019-10-11T00:00:00Zhttps://blog.meain.io/2019/making-sure-you-wont-commit-conflict-markers/<p>Recently I made a mistake of committing a conflict marker and pushing that code to Github.</p>
<p>That is when I thought that I could have a easily avoided this if I had added a <code>git-hook</code> to warn me if the code that I
commit had conflict markers. Here is how you would set up something like that.</p>
<h1 id="what-are-git-hooks%3F" tabindex="-1">What are git-hooks? <a class="direct-link" href="https://blog.meain.io/2019/making-sure-you-wont-commit-conflict-markers/#what-are-git-hooks%3F">#</a></h1>
<p>In case you are new to <code>git-hooks</code>, it is a way by which git lets to hook into the different things that git does.
You can write <code>git-hooks</code> to do a lot of things. You can modify a commit message to add some additional info using a git
hook. You can print out extra stuff on commit(what we will be using) or push or any such operation.</p>
<p>Git hooks are small executables that you place in <code>.git/hooks</code> directory of your repo. If you were to check your
<code>.git/hooks</code> directory right now for any repo, you can see a few files which ends with <code>*.sample</code>. Those are samples
that git by default puts there so that you can see what it can probably do.</p>
<h1 id="our-solution" tabindex="-1">Our solution <a class="direct-link" href="https://blog.meain.io/2019/making-sure-you-wont-commit-conflict-markers/#our-solution">#</a></h1>
<p>What we are going to do is to create a pre-commit hook which will show something like this.
It will give us a big bold red warning when we try to commit a piece of code that contains conflict markers in it.</p>
<p><img src="https://blog.meain.io/img/conflict-marker.png" alt="conflict-marker" /></p>
<h3 id="tldr%3B" tabindex="-1">tldr; <a class="direct-link" href="https://blog.meain.io/2019/making-sure-you-wont-commit-conflict-markers/#tldr%3B">#</a></h3>
<p>Place this piece of code as <code>.git/hook/pre-commit</code> and make it executable using <code>chmod +x .git/hooks/pre-commit</code></p>
<pre class="language-shell"><code class="language-shell"><span class="token shebang important">#! /bin/sh</span><br /><br /><span class="token assign-left variable">RED</span><span class="token operator">=</span><span class="token variable"><span class="token variable">$(</span>tput setab <span class="token number">1</span><span class="token variable">)</span></span><br /><span class="token assign-left variable">NORMAL</span><span class="token operator">=</span><span class="token variable"><span class="token variable">$(</span>tput sgr0<span class="token variable">)</span></span><br /><br /><span class="token assign-left variable">CONFLICT_MARKERS</span><span class="token operator">=</span><span class="token string">'<<<<<<<|=======|>>>>>>>'</span><br /><span class="token assign-left variable">CHECK</span><span class="token operator">=</span><span class="token variable"><span class="token variable">$(</span><span class="token function">git</span> <span class="token function">diff</span> <span class="token parameter variable">--staged</span> <span class="token operator">|</span> <span class="token function">grep</span> <span class="token string">"^+"</span> <span class="token operator">|</span> <span class="token function">grep</span> <span class="token parameter variable">-Ei</span> <span class="token string">"<span class="token variable">$CONFLICT_MARKERS</span>"</span> <span class="token parameter variable">-c</span><span class="token variable">)</span></span><br /><span class="token keyword">if</span> <span class="token punctuation">[</span> <span class="token string">"<span class="token variable">$CHECK</span>"</span> <span class="token parameter variable">-gt</span> <span class="token number">0</span> <span class="token punctuation">]</span><br /><span class="token keyword">then</span><br /> <span class="token builtin class-name">echo</span> <span class="token string">"<span class="token variable">$RED</span> WARNING <span class="token variable">$NORMAL</span> Conflict markers sill preset"</span><br /> <span class="token function">git</span> <span class="token function">diff</span> --name-only -G<span class="token string">"<span class="token variable">$CONFLICT_MARKERS</span>"</span><br /> <span class="token comment"># uncomment the below line if you need the commit to not go through at all</span><br /> <span class="token comment"># exit 1</span><br /><span class="token keyword">fi</span></code></pre>
<h3 id="explanation" tabindex="-1">Explanation <a class="direct-link" href="https://blog.meain.io/2019/making-sure-you-wont-commit-conflict-markers/#explanation">#</a></h3>
<p>OK, that might look like a mess, but let me explain.</p>
<p>After the usual <code>shebang</code>, we define a variable called <code>CONFLICT_MARKERS</code> with the possible conflict markers separated
by <code>|</code>. The reason why we are separating the markers by <code>|</code> is because we use them in <code>grep</code> and in <code>regex</code> land it
means, match one of <code><<<<<<<</code>, <code>=======</code>, <code>>>>>>>></code>.</p>
<p>On the next line we do the actual check to see if the conflict markers are present in the code using <code>grep</code>. In this
line what we do is we pipe the diff of staged commits through <code>grep '^+'</code> which will filter out only the lines which
were added and that in turn gets passed to the next <code>grep</code> which counts the number of times any of those conflict markers
appear in the added lines in the diff and store the count to a variable <code>CHECK</code>.</p>
<p>Next line is an <code>if</code> check to see if the value in variable <code>CHECK</code> is greater than <code>0</code>.
And if it is greater than <code>0</code>, we show the warning and a list of files which has conflict markers in them.</p>
<p>You can prevent the commit from ever happening if you uncomment <code>exit 1</code>. But I would personally not recommend this
unless you add a way to bypass this, maybe using something like and env variable. In some rare cases where you might
need one of those conflict markers in your code, you will not be able to make the commit without modifying this code.</p>
<h3 id="usage" tabindex="-1">Usage <a class="direct-link" href="https://blog.meain.io/2019/making-sure-you-wont-commit-conflict-markers/#usage">#</a></h3>
<p>As mentioned above, you can put this code in <code>.git/hooks/pre-commit</code> of any git repo where you need this check.</p>
<p>You can also put this in <code>$HOME/.git_template/hooks/pre-commit</code> to enable this is all the new git repos you create. Any
hooks you place here will be copied over to the hooks dir in the new git repos you create.</p>
How to create a SVG blob thingy in HTML Canvas2019-11-12T00:00:00Zhttps://blog.meain.io/2019/canvas-svg-blob/<p>You might have seen a lot of colorful blobs everywhere these days.
It is simple and looks pretty without much effort.
You even have tools like <a href="https://www.blobmaker.app/">blobmaker.app</a> to help you easily create blobs.
Let me show you how you can code up your own svg blob thingy.</p>
<p>Here is what we will be building.</p>
<p><img src="https://i.imgur.com/69tltSr.gif" alt="gif" /></p>
<blockquote>
<p><a href="https://github.com/meain/svg-blob">github</a> and <a href="https://codesandbox.io/s/admiring-hellman-m4dre">codesandbox</a></p>
</blockquote>
<p>OK, here we go.</p>
<h3 id="1.-prepare-environment" tabindex="-1">1. Prepare environment <a class="direct-link" href="https://blog.meain.io/2019/canvas-svg-blob/#1.-prepare-environment">#</a></h3>
<p>Below is the basic setup. Once this is out of the way, we will get to the actual code.</p>
<h4 id="html" tabindex="-1">HTML <a class="direct-link" href="https://blog.meain.io/2019/canvas-svg-blob/#html">#</a></h4>
<pre class="language-html"><code class="language-html"><span class="token doctype"><span class="token punctuation"><!</span><span class="token doctype-tag">DOCTYPE</span> <span class="token name">html</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>html</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>head</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>meta</span> <span class="token attr-name">charset</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>UTF-8<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>meta</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>viewport<span class="token punctuation">"</span></span> <span class="token attr-name">content</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>width=device-width,initial-scale=1<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>title</span><span class="token punctuation">></span></span>Blob<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>title</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>link</span> <span class="token attr-name">rel</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>stylesheet<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>style.css<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>text/css<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>head</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>body</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>blob.js<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script"></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>body</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>html</span><span class="token punctuation">></span></span></code></pre>
<h4 id="css" tabindex="-1">CSS <a class="direct-link" href="https://blog.meain.io/2019/canvas-svg-blob/#css">#</a></h4>
<pre class="language-css"><code class="language-css"><span class="token selector">body,<br />html</span> <span class="token punctuation">{</span><br /> <span class="token property">margin</span><span class="token punctuation">:</span> 0<span class="token punctuation">;</span><br /> <span class="token property">width</span><span class="token punctuation">:</span> 100vw<span class="token punctuation">;</span><br /> <span class="token property">height</span><span class="token punctuation">:</span> 100vh<span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><br /><span class="token selector">canvas</span> <span class="token punctuation">{</span><br /> <span class="token property">position</span><span class="token punctuation">:</span> fixed<span class="token punctuation">;</span><br /> <span class="token property">touch-action</span><span class="token punctuation">:</span> none<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<h4 id="javascript" tabindex="-1">Javascript <a class="direct-link" href="https://blog.meain.io/2019/canvas-svg-blob/#javascript">#</a></h4>
<pre class="language-javascript"><code class="language-javascript"><span class="token keyword">function</span> <span class="token function">create</span><span class="token punctuation">(</span><span class="token parameter">root</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> canvas <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"canvas"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> canvas<span class="token punctuation">.</span><span class="token function">setAttribute</span><span class="token punctuation">(</span><span class="token string">"touch-action"</span><span class="token punctuation">,</span> <span class="token string">"none"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> root<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>canvas<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">const</span> <span class="token function-variable function">resize</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> canvas<span class="token punctuation">.</span>width <span class="token operator">=</span> window<span class="token punctuation">.</span>innerWidth<span class="token punctuation">;</span><br /> canvas<span class="token punctuation">.</span>height <span class="token operator">=</span> window<span class="token punctuation">.</span>innerHeight<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /> window<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">"resize"</span><span class="token punctuation">,</span> resize<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">resize</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> canvas<span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">const</span> canvas <span class="token operator">=</span> <span class="token function">create</span><span class="token punctuation">(</span>document<span class="token punctuation">.</span>body<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// get the canvas</span><br /><span class="token keyword">const</span> ctx <span class="token operator">=</span> canvas<span class="token punctuation">.</span><span class="token function">getContext</span><span class="token punctuation">(</span><span class="token string">"2d"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// initialize for 2d</span></code></pre>
<h3 id="2.-drawing-the-svg-blob" tabindex="-1">2. Drawing the svg-blob <a class="direct-link" href="https://blog.meain.io/2019/canvas-svg-blob/#2.-drawing-the-svg-blob">#</a></h3>
<p>The basic idea with drawing and <code>svg-blob</code> is that we have a circle and we are finding some points which lie inside
or outside of the circle and using them to draw a <a href="https://javascript.info/bezier-curve">Bézier curve</a>.</p>
<p><strong>A quick intro to how Bézier curves work.</strong> We have a total of 4 points, out of which 2 will say where to start and where
to end and two will say how to curve that line.</p>
<p>In our specific case, we find these points and draw an SVG path with a solid fill.
We have simplified drawing the SVG path by using a <code>smooth curve (S)</code> instead of a <code>curve (C)</code>.
This is defined in the <a href="https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths">SVG path spec</a> and can be used when the
points used to curve the lines are reflections of each other.</p>
<p>I am guessing that you have got the idea that we will need multiple sets of 3 points, <code>start</code>, <code>end</code> and
<code>the thing that is used to curve the line</code>. Actually we are only getting 2 points, just the <code>end</code> and <code>curving thingy</code>
because <code>start</code> is considered as the where the previous command ended.</p>
<p><strong>A blob is just a collection of curved arcs whose inner area is filed with color.</strong></p>
<p>The code for this looks something like this.</p>
<blockquote>
<p><code>rint</code> just returns a random positive integer and <code>getRandomBetween</code> return a random positive or negative number
between that range.</p>
</blockquote>
<pre class="language-javascript"><code class="language-javascript"><span class="token comment">// Getting points needed to draw the circle</span><br /><span class="token keyword">function</span> <span class="token function">getCirclePoints</span><span class="token punctuation">(</span><span class="token parameter">base<span class="token punctuation">,</span> radius</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> angles <span class="token operator">=</span> <span class="token punctuation">[</span> <span class="token comment">// angles at which we compute points</span><br /> <span class="token function">rint</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">90</span> <span class="token operator">-</span> <span class="token number">45</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token function">rint</span><span class="token punctuation">(</span><span class="token number">90</span><span class="token punctuation">,</span> <span class="token number">180</span> <span class="token operator">-</span> <span class="token number">45</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token function">rint</span><span class="token punctuation">(</span><span class="token number">180</span><span class="token punctuation">,</span> <span class="token number">270</span> <span class="token operator">-</span> <span class="token number">45</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token function">rint</span><span class="token punctuation">(</span><span class="token number">270</span><span class="token punctuation">,</span> <span class="token number">360</span> <span class="token operator">-</span> <span class="token number">45</span><span class="token punctuation">)</span><br /> <span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token keyword">const</span> positions <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">let</span> a <span class="token keyword">in</span> angles<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> angle <span class="token operator">=</span> <span class="token punctuation">(</span>angles<span class="token punctuation">[</span>a<span class="token punctuation">]</span> <span class="token operator">*</span> Math<span class="token punctuation">.</span><span class="token constant">PI</span><span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token number">180</span><span class="token punctuation">;</span><br /> <span class="token keyword">let</span> ba <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">(</span>angles<span class="token punctuation">[</span>a<span class="token punctuation">]</span> <span class="token operator">-</span> <span class="token number">20</span><span class="token punctuation">)</span> <span class="token operator">*</span> Math<span class="token punctuation">.</span><span class="token constant">PI</span><span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token number">180</span><span class="token punctuation">;</span><br /> <span class="token keyword">let</span> rr <span class="token operator">=</span> radius <span class="token operator">+</span> <span class="token function">getRandomBetween</span><span class="token punctuation">(</span><span class="token number">40</span><span class="token punctuation">,</span> <span class="token number">100</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> positions<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span><span class="token punctuation">{</span><br /> <span class="token literal-property property">x</span><span class="token operator">:</span> base<span class="token punctuation">.</span>x <span class="token operator">+</span> radius <span class="token operator">*</span> Math<span class="token punctuation">.</span><span class="token function">sin</span><span class="token punctuation">(</span>angle<span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">y</span><span class="token operator">:</span> base<span class="token punctuation">.</span>y <span class="token operator">+</span> radius <span class="token operator">*</span> Math<span class="token punctuation">.</span><span class="token function">cos</span><span class="token punctuation">(</span>angle<span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">mx</span><span class="token operator">:</span> base<span class="token punctuation">.</span>x <span class="token operator">+</span> rr <span class="token operator">*</span> Math<span class="token punctuation">.</span><span class="token function">sin</span><span class="token punctuation">(</span>ba<span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">my</span><span class="token operator">:</span> base<span class="token punctuation">.</span>y <span class="token operator">+</span> rr <span class="token operator">*</span> Math<span class="token punctuation">.</span><span class="token function">cos</span><span class="token punctuation">(</span>ba<span class="token punctuation">)</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> positions<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span>positions<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// last one is same as first to make sure they line up</span><br /> <span class="token keyword">return</span> positions<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<blockquote>
<p>angle has to dealt with in radians</p>
</blockquote>
<p>Now we have to create the <code>d</code> thingy for the path SVG element. This defines how to draw the path.
There are a lot of notation used for the <code>d</code> thingy, but the ones that you have to know are.</p>
<ul>
<li><code>M</code>: move to the specified position</li>
<li><code>S</code>: smooth curve, takes 2 points and draw a bezier curve</li>
<li><code>Z</code>: close the loop</li>
</ul>
<pre class="language-javascript"><code class="language-javascript"><span class="token keyword">function</span> <span class="token function">drawPath</span><span class="token punctuation">(</span><span class="token parameter">points</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> cpath <span class="token operator">=</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">M</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>points<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>x<span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">,</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>points<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">.</span>y<span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span><br /> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">let</span> point <span class="token keyword">of</span> points<span class="token punctuation">)</span><br /> cpath <span class="token operator">+=</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">S</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>point<span class="token punctuation">.</span>mx<span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">,</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>point<span class="token punctuation">.</span>my<span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">,</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>point<span class="token punctuation">.</span>x<span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">,</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>point<span class="token punctuation">.</span>y<span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span><br /> cpath <span class="token operator">+=</span> <span class="token string">"Z"</span><span class="token punctuation">;</span><br /> <span class="token keyword">let</span> p <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Path2D</span><span class="token punctuation">(</span>cpath<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> ctx<span class="token punctuation">.</span><span class="token function">clearRect</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> canvas<span class="token punctuation">.</span>width<span class="token punctuation">,</span> canvas<span class="token punctuation">.</span>height<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> ctx<span class="token punctuation">.</span>fillStyle <span class="token operator">=</span> <span class="token string">"rgb(229, 244, 216)"</span><span class="token punctuation">;</span><br /> ctx<span class="token punctuation">.</span><span class="token function">fill</span><span class="token punctuation">(</span>p<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>With this in place, all we need to do is.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="token function">drawPath</span><span class="token punctuation">(</span><span class="token function">getCirclePoints</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">x</span><span class="token operator">:</span> <span class="token number">300</span><span class="token punctuation">,</span> <span class="token literal-property property">y</span><span class="token operator">:</span> <span class="token number">300</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">150</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>A sample path will look something like this.</p>
<pre class="language-xml"><code class="language-xml"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>svg</span><br /><span class="token attr-name">width</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>600<span class="token punctuation">"</span></span><br /><span class="token attr-name">height</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>600<span class="token punctuation">"</span></span><br /><span class="token attr-name">xmlns</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>http://www.w3.org/2000/svg<span class="token punctuation">"</span></span><br /><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>path</span> <span class="token attr-name">d</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>M175.7,-137.1C200.2,-109.3,173.7,-37.9,150.8,23C127.8,84,108.4,134.3,74.6,147.8C40.9,161.3,-7.1,137.8,-54.7,114.6C-102.4,91.3,-149.7,68.1,-175.8,20.9C-201.9,-26.4,-206.9,-97.7,-174.1,-127.3C-141.3,-157,-70.6,-145,2.5,-147C75.6,-148.9,151.2,-164.9,175.7,-137.1Z<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>svg</span><span class="token punctuation">></span></span></code></pre>
<p>With all that in place, you get something this, a simple blob.
Here is the code for the static one in <a href="https://codesandbox.io/s/tender-darkness-ujk1u">codesandbox</a>.</p>
<p><img src="https://blog.meain.io/img/blob.png" alt="blob" /></p>
<h3 id="3.-animating-the-svg-blob" tabindex="-1">3. Animating the svg-blob <a class="direct-link" href="https://blog.meain.io/2019/canvas-svg-blob/#3.-animating-the-svg-blob">#</a></h3>
<p>Now to animate the svg blob, we can make use of a library called <a href="https://github.com/tweenjs/es6-tween"><code>es6-tween</code></a>.
This is just an <code>es6</code> rewrite of the <a href="http://createjs.com/tweenjs"><code>tweenjs</code></a> library.</p>
<p>Essentially we will be draw a new blob every 2 seconds and tween between them.
We create a <code>drawBlob</code> function like this ... and we call it in a <code>setTimeoute</code>. It is that simple.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="token keyword">import</span> <span class="token punctuation">{</span> Tween<span class="token punctuation">,</span> autoPlay <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"es6-tween"</span><span class="token punctuation">;</span><br /><span class="token function">autoPlay</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">let</span> prevPoints <span class="token operator">=</span> <span class="token keyword">null</span><span class="token punctuation">;</span><br /><span class="token keyword">function</span> <span class="token function">drawBlob</span><span class="token punctuation">(</span><span class="token parameter">ctx</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> points <span class="token operator">=</span> <span class="token function">getCirclePoints</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">x</span><span class="token operator">:</span> <span class="token number">300</span><span class="token punctuation">,</span> <span class="token literal-property property">y</span><span class="token operator">:</span> <span class="token number">300</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">120</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>prevPoints <span class="token operator">===</span> <span class="token keyword">null</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">drawPath</span><span class="token punctuation">(</span>ctx<span class="token punctuation">,</span> points<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> coords <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token operator">...</span>prevPoints<span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token keyword">new</span> <span class="token class-name">Tween</span><span class="token punctuation">(</span>coords<span class="token punctuation">)</span><br /> <span class="token punctuation">.</span><span class="token function">to</span><span class="token punctuation">(</span>points<span class="token punctuation">,</span> <span class="token number">1000</span><span class="token punctuation">)</span><br /> <span class="token punctuation">.</span><span class="token function">on</span><span class="token punctuation">(</span><span class="token string">"update"</span><span class="token punctuation">,</span> <span class="token parameter">p</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token function">drawPath</span><span class="token punctuation">(</span>ctx<span class="token punctuation">,</span> p<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><br /> <span class="token punctuation">.</span><span class="token function">start</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> prevPoints <span class="token operator">=</span> points<span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><br /><span class="token function">drawBlob</span><span class="token punctuation">(</span>ctx<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token function">setInterval</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token function">drawBlob</span><span class="token punctuation">(</span>ctx<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">2000</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>And there you go, hope that made some sense.</p>
Table operations like in Nu shell in BASH2019-11-16T00:00:00Zhttps://blog.meain.io/2019/nu-like-table-operations-in-bash/<p>Hi, another installment of why people should just write more bash.
I am not sure if you are aware of it. There is a new shell written in Rust called <a href="https://www.nushell.sh/">Nu Shell</a>.
One of the main ideas of it is that you treat every data as table data and then work on top if as if
you were working with SQL queries.</p>
<p>I am not a big fan of this approach personally. I would stick to treating data as a text stream than a tabular list.
Sure, it will let you put some abstraction on top of which might make your life a bit simpler in some case, but you are
missing out the power of just treating is as a simple text stream.</p>
<p>I was inspired to write this blog after seeing this <a href="https://twitter.com/nixcraft/status/1195665338416787456?s=09">tweet</a> from nixcraft.
You can do a lot of stuff listed in <a href="https://book.nushell.sh/en/working_with_tables">working with tables in nushell</a> right in BASH.</p>
<p>I don't wanna BASH(pun intended) Nu shell. A lot of people seem to love it, plus Nu helps with making items into a table
in the first place easier. I just want people to understand that you don't need Nu shell to do a lot of the things that
they are mentioning.</p>
<p>OK, enough blabbering, lets get to the actual content.</p>
<p>I just decided to take a few examples that they have in their <a href="https://book.nushell.sh/">book</a> and show how you would
achieve the same thing in BASH.</p>
<p>I am gonna use <code>ls</code> as the data input here as they seem to use it for their demo in <a href="https://book.nushell.sh/en/working_with_tables">working with tables</a>.
In <code>ls</code>, the data is already in columns, so I guess you could say that it already looks like a table but without
borders (lets tare down the borders).</p>
<h1 id="basics" tabindex="-1">Basics <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#basics">#</a></h1>
<h2 id="setting-baseline---bare-bones-ls-(not-exactly)" tabindex="-1">Setting baseline - bare bones <code>ls</code> (not exactly) <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#setting-baseline---bare-bones-ls-(not-exactly)">#</a></h2>
<p>This is the output from my <code>blog</code> directly. Its in <code>jekyll</code>, nothing fancy.</p>
<pre><code>total 64
-rw-r--r-- 1 meain staff 1077 Aug 10 17:59 LICENSE.md
-rw-r--r-- 1 meain staff 139 May 27 2017 README.md
-rw-r--r-- 1 meain staff 1605 Aug 10 18:47 _config.yml
drwxr-xr-x 13 meain staff 416 Nov 13 10:56 _includes
drwxr-xr-x 6 meain staff 192 Nov 12 21:43 _layouts
drwxr-xr-x 26 meain staff 832 Nov 16 17:49 _posts
drwxr-xr-x 12 meain staff 384 Nov 16 17:15 _site
-rw-r--r-- 1 meain staff 533 Aug 10 17:59 about.md
drwxr-xr-x 6 meain staff 192 Mar 20 2018 assets
-rwxr-xr-x 1 meain staff 2166 Nov 13 10:55 favicon.png
-rw-r--r-- 1 meain staff 1291 May 23 2017 feed.xml
-rw-r--r-- 1 meain staff 863 Aug 27 2017 index.html
-rwxr-xr-x 1 meain staff 13 Oct 11 20:16 run-dev
</code></pre>
<p>Actually we are going to use <code>ls | tail -n+2</code> from here on out as there is that <code>total 64</code> thingy on top and we need to
get rid of that when working with "table" ish data.</p>
<p>You might know about <code>tail</code> already but if you pass in a value like <code>tail -n+<value></code> it will actually show everything
except that many <code>lines-1</code>. So with that, my output will look something like this.</p>
<pre><code>-rw-r--r-- 1 meain staff 1077 Aug 10 17:59 LICENSE.md
-rw-r--r-- 1 meain staff 139 May 27 2017 README.md
-rw-r--r-- 1 meain staff 1605 Aug 10 18:47 _config.yml
drwxr-xr-x 13 meain staff 416 Nov 13 10:56 _includes
drwxr-xr-x 6 meain staff 192 Nov 12 21:43 _layouts
drwxr-xr-x 26 meain staff 832 Nov 16 17:53 _posts
drwxr-xr-x 12 meain staff 384 Nov 16 17:15 _site
-rw-r--r-- 1 meain staff 533 Aug 10 17:59 about.md
drwxr-xr-x 6 meain staff 192 Mar 20 2018 assets
-rwxr-xr-x 1 meain staff 2166 Nov 13 10:55 favicon.png
-rw-r--r-- 1 meain staff 1291 May 23 2017 feed.xml
-rw-r--r-- 1 meain staff 863 Aug 27 2017 index.html
-rwxr-xr-x 1 meain staff 13 Oct 11 20:16 run-dev
</code></pre>
<p>btw a Nu shell output of <code>ls</code> will look something like this</p>
<pre><code>---+---------------+------+----------+---------+------------+------------
# | name | type | readonly | size | accessed | modified
---+---------------+------+----------+---------+------------+------------
0 | add.rs | File | | 2.7 KB | 2 days ago | 2 days ago
1 | sum.rs | File | | 3.0 KB | 2 days ago | 2 days ago
2 | inc.rs | File | | 11.8 KB | 2 days ago | 2 days ago
3 | str.rs | File | | 21.4 KB | 2 days ago | 2 days ago
4 | skip.rs | File | | 1.7 KB | 2 days ago | 2 days ago
5 | textview.rs | File | | 9.4 KB | 2 days ago | 2 days ago
6 | binaryview.rs | File | | 13.0 KB | a day ago | a day ago
7 | edit.rs | File | | 2.7 KB | 2 days ago | 2 days ago
8 | tree.rs | File | | 3.0 KB | 2 days ago | 2 days ago
9 | sys.rs | File | | 9.2 KB | 2 days ago | 2 days ago
---+---------------+------+----------+---------+------------+------------
</code></pre>
<blockquote>
<p>For the below commands, assume I am doing <code>ls -l | tail -n+2</code> when I type <code>ls</code></p>
</blockquote>
<h2 id="selecting-specific-columns" tabindex="-1">Selecting specific columns <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#selecting-specific-columns">#</a></h2>
<p>Lets say you just wanna see only the name and the size, what you can do is</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ls</span> <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'{print $9,$5}'</span> <span class="token comment"># bash</span><br /><span class="token function">ls</span> <span class="token operator">|</span> pick name size <span class="token comment"># nu</span></code></pre>
<p>What this does is that it selects the 9th and 5th column from the above output separated by spaces.
The output of this will look kinda like below.</p>
<pre><code>LICENSE.md 1077
README.md 139
_config.yml 1605
_includes 416
_layouts 192
_posts 832
_site 384
about.md 533
assets 192
favicon.png 2166
feed.xml 1291
index.html 863
run-dev 13
</code></pre>
<p>Now, if you need this in column format you can just pipe the output into <code>column</code></p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ls</span> <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'{print $9,$5}'</span> <span class="token operator">|</span> <span class="token function">column</span> <span class="token parameter variable">-t</span><br /><span class="token function">ls</span> <span class="token operator">|</span> pick name size <span class="token comment"># nu</span></code></pre>
<p>and you output will become</p>
<pre><code>LICENSE.md 1077
README.md 139
_config.yml 1605
_includes 416
_layouts 192
_posts 832
_site 384
about.md 533
assets 192
favicon.png 2166
feed.xml 1291
index.html 863
run-dev 13
</code></pre>
<blockquote>
<p>caveat: if there are spaces in filenames, this will not work</p>
</blockquote>
<h2 id="sorting-the-data" tabindex="-1">Sorting the data <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#sorting-the-data">#</a></h2>
<p>OK, lets say we need to sort the files by size or by name. What would you do?
For both the use cases you can make use of the <code>sort</code> command.</p>
<p>If you need to sort by size, you just give the following command.</p>
<ul>
<li><code>-n</code>: say that it is a numerical sort</li>
<li><code>-k</code>: specify which column to use for sort</li>
</ul>
<pre class="language-shell"><code class="language-shell"><span class="token function">ls</span> <span class="token operator">|</span> <span class="token function">sort</span> <span class="token parameter variable">-nk5</span> <span class="token comment"># bash</span><br /><span class="token function">ls</span> <span class="token operator">|</span> sort-by size <span class="token comment"># nu</span></code></pre>
<p>This will give you something like the following</p>
<pre><code>-rwxr-xr-x 1 meain staff 13 Oct 11 20:16 run-dev
-rw-r--r-- 1 meain staff 139 May 27 2017 README.md
drwxr-xr-x 6 meain staff 192 Mar 20 2018 assets
drwxr-xr-x 6 meain staff 192 Nov 12 21:43 _layouts
drwxr-xr-x 12 meain staff 384 Nov 16 17:15 _site
drwxr-xr-x 13 meain staff 416 Nov 13 10:56 _includes
-rw-r--r-- 1 meain staff 533 Aug 10 17:59 about.md
drwxr-xr-x 26 meain staff 832 Nov 16 17:59 _posts
-rw-r--r-- 1 meain staff 863 Aug 27 2017 index.html
-rw-r--r-- 1 meain staff 1077 Aug 10 17:59 LICENSE.md
-rw-r--r-- 1 meain staff 1291 May 23 2017 feed.xml
-rw-r--r-- 1 meain staff 1605 Aug 10 18:47 _config.yml
-rwxr-xr-x 1 meain staff 2166 Nov 13 10:55 favicon.png
</code></pre>
<p>You could the probably just pick only the name and size from it. The command for that would be same as above.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ls</span> <span class="token operator">|</span> <span class="token function">sort</span> <span class="token parameter variable">-nk5</span> <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'{print $9,$5}'</span> <span class="token operator">|</span> <span class="token function">column</span> <span class="token parameter variable">-t</span> <span class="token comment"># bash</span><br /><span class="token function">ls</span> <span class="token operator">|</span> sort-by size <span class="token operator">|</span> pick name size <span class="token comment"># nu</span></code></pre>
<p>and this would give you</p>
<pre><code>run-dev 13
README.md 139
assets 192
_layouts 192
_site 384
_includes 416
about.md 533
_posts 832
index.html 863
LICENSE.md 1077
feed.xml 1291
_config.yml 1605
favicon.png 2166
</code></pre>
<p><strong>Pretty neat, huh?</strong></p>
<p>Instead if you where to sort by name, you would have one less parameter to sort. You will not have the <code>-n</code> flag as it
is not a numerical column.</p>
<p>So, if you need to do that, the command would be:</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ls</span> <span class="token operator">|</span> <span class="token function">sort</span> <span class="token parameter variable">-k9</span> <span class="token comment"># bash</span><br /><span class="token function">ls</span> <span class="token operator">|</span> sort-by name <span class="token comment"># nu</span></code></pre>
<h2 id="first-and-skip" tabindex="-1"><code>first</code> and <code>skip</code> <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#first-and-skip">#</a></h2>
<p>Well, this is pretty much <code>head</code> and <code>tail</code>.</p>
<ul>
<li>To take the first n items, you would to <code>head -n<number></code></li>
<li>To take the last n items, you would do <code>tail -n<number></code></li>
<li>To skip the first n items, you would do <code>tail -n+<number+1></code></li>
</ul>
<p>As simple as that. Lets say you wanna sort by size and take the first 5 and skip the first two after that.
It would look something like this.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ls</span> <span class="token operator">|</span> <span class="token function">sort</span> <span class="token parameter variable">-k9</span> <span class="token operator">|</span> <span class="token function">head</span> <span class="token parameter variable">-n5</span> <span class="token operator">|</span> <span class="token function">tail</span> -n+3 <span class="token comment"># bash</span><br /><span class="token function">ls</span> <span class="token operator">|</span> sort-by size <span class="token operator">|</span> first <span class="token number">5</span> <span class="token operator">|</span> skip <span class="token number">2</span> <span class="token comment"># nu</span></code></pre>
<blockquote>
<p>If you really wanna go there you can always use <code>sed 11q</code> instead of <code>head</code>. One for the memes.</p>
</blockquote>
<h2 id="picking-the-nth-item" tabindex="-1">Picking the <code>nth</code> item <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#picking-the-nth-item">#</a></h2>
<p>Nu shell has this command called <code>nth</code> which lets you pick the nth line in a list.
You can do something similar by using awk.</p>
<p>Lets say you wanna pick the 5th element</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ls</span> <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'5 == NR'</span> <span class="token comment"># bash</span><br /><span class="token function">ls</span> <span class="token operator">|</span> nth <span class="token number">5</span></code></pre>
<p>but awk is actually even more powerful, lets say you wanna pick every even item, you can do</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ls</span> <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'0 == NR % 2'</span> <span class="token comment"># bash</span></code></pre>
<blockquote>
<p><code>NR</code> in awk is a special variable, it give you the current line number. You can do any math operation on it.
In here we just take the modulus of it with 2 and prints if it is 0.</p>
</blockquote>
<h2 id="check-if-column-value-greater-than-a-specific-value" tabindex="-1">Check if column value greater than a specific value <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#check-if-column-value-greater-than-a-specific-value">#</a></h2>
<p>Nu shell provides this command called <code>where</code> which you can use to compare values like an SQL where clause.
You can actually do that with awk.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ls</span> <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'$5 > 1000'</span> <span class="token comment"># bash</span><br /><span class="token function">ls</span> <span class="token operator">|</span> where size <span class="token operator">></span> <span class="token number">1000</span> <span class="token comment"># nu</span></code></pre>
<p>Ideally the command would look more like</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ls</span> <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'$5 > 1000 {print $0}'</span></code></pre>
<p>but the <code>{print $0}</code> part will be put there if we do not have anything in there.</p>
<h2 id="sum-up-values-in-a-column" tabindex="-1">Sum up values in a column <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#sum-up-values-in-a-column">#</a></h2>
<p>Again, you can just use awk for this. Maybe this is a blog about awk than bash?</p>
<p>So, lets say you wanna view how much cpu each process takes and sum it up.
If you run the ps command it will give you this data. For our use let us just get the command name and the cpu usage.</p>
<p>Running</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ps</span> axhc <span class="token parameter variable">-o</span> command,%cpu <span class="token parameter variable">-r</span></code></pre>
<p>will give you something like the following</p>
<pre><code>COMMAND %CPU
firefox 67.3
-zsh 13.0
WindowServer 6.9
hidd 1.4
plugin-container 1.1
tmux 0.9
karabiner_grabbe 0.9
...
</code></pre>
<p>Now, if we needed to sum up the values of cpu, we can use the following command</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">ps</span> axhc <span class="token parameter variable">-o</span> command,%cpu <span class="token parameter variable">-r</span> <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'{s+=$2} END {print s}'</span></code></pre>
<p>OK, let me explain. We just create a variable <code>s</code> and add all the values of <code>$2</code> which is the second column item (cpu)
and once we are done, we print out <code>s</code>. Its as simple as that.</p>
<h1 id="meta" tabindex="-1">Meta <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#meta">#</a></h1>
<h2 id="what-if-the-separator-is-not-space%3F" tabindex="-1">What if the separator is not space? <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#what-if-the-separator-is-not-space%3F">#</a></h2>
<p>Well, in most common cases, the separator will be space. But what if it is not?
You can pass in a argument to most of the commands to say what the separator is.</p>
<p>I am going to use a file which looks like below for demo. Just pulled from their page.</p>
<pre><code>Octavia | Butler | Writer
Bob | Ross | Painter
Antonio | Vivaldi | Composer
</code></pre>
<h3 id="for-column-you-can-use--s." tabindex="-1">For <code>column</code> you can use <code>-s</code>. <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#for-column-you-can-use--s.">#</a></h3>
<p>If you wanna see this in columns, you can just do</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">column</span> -s<span class="token string">'|'</span> <span class="token parameter variable">-t</span> people.txt <span class="token comment"># bash</span><br /><span class="token function">open</span> people.txt <span class="token operator">|</span> lines <span class="token operator">|</span> split-column <span class="token string">"|"</span> <span class="token comment"># nu</span></code></pre>
<p>The output for this will be like what you would expect for it.</p>
<pre><code>Octavia Butler Writer
Bob Ross Painter
Antonio Vivaldi Composer
</code></pre>
<h3 id="for-awk-you-have--f" tabindex="-1">For <code>awk</code> you have <code>-F</code> <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#for-awk-you-have--f">#</a></h3>
<p>You can just pass the separator using the flag <code>-F</code>.</p>
<p>Lets say you just wanna view to the first column, you can just do.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">awk</span> -F<span class="token string">'|'</span> <span class="token string">'{print $1}'</span> people.txt</code></pre>
<p>The output will be, as you expect</p>
<pre><code>Octavia
Bob
Antonio
</code></pre>
<p>Just as a reminder, this is not the only way to do it. Here are a few other possible ways.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">cut</span> -d<span class="token string">'|'</span> <span class="token parameter variable">-f1</span> people.txt<br /><span class="token function">sed</span> <span class="token string">'s/\ |.*//'</span> people.txt<br /><span class="token function">grep</span> <span class="token parameter variable">-o</span> <span class="token string">"^\w*<span class="token entity" title="\b">\b</span>"</span> people.txt<br /><span class="token function">grep</span> <span class="token parameter variable">-Eo</span> <span class="token string">'^[^|]+'</span> people.txt</code></pre>
<h3 id="for-sort-you-have--t" tabindex="-1">For <code>sort</code> you have <code>-t</code> <a class="direct-link" href="https://blog.meain.io/2019/nu-like-table-operations-in-bash/#for-sort-you-have--t">#</a></h3>
<p>If you wanna sort by the third column, you can run</p>
<pre><code>sort -t '|' -k3 people.txt
</code></pre>
<p>you will essentially get you</p>
<pre><code>Antonio | Vivaldi | Composer
Bob | Ross | Painter
Octavia | Butler | Writer
</code></pre>
<p>Now pass it to column for a cleaner output</p>
<pre><code>sort -t '|' -k3 people.txt | column -s'|' -t
</code></pre>
<p>and that will give you</p>
<pre><code>Antonio Vivaldi Composer
Bob Ross Painter
Octavia Butler Writer
</code></pre>
<hr />
<p>And that is a wrap, I guess I convinced you that BASH is really powerful.
Hopefully, I got at least a few more people to write more BASH.</p>
Switching to DuckDuckGo2019-12-17T00:00:00Zhttps://blog.meain.io/2019/switching-to-duckduckgo/<p>So yeah, recently I decided to switch from Google to DuckDuckGo.
DuckDuckGo is great, don't get me wrong. But at times, Google just has better results.
I usually find myself going back to Google for a <em>max</em> of 10%-15% searches.</p>
<blockquote>
<p>DuckDuckGo is really good, just not creepy good.</p>
</blockquote>
<h1 id="if-duckduckgo-fails" tabindex="-1">If DuckDuckGo fails <a class="direct-link" href="https://blog.meain.io/2019/switching-to-duckduckgo/#if-duckduckgo-fails">#</a></h1>
<p>I am a lazy person, if I have to do the search two times I will be pissed.
So, I was thinking about having something that will let me quickly search for the same thing in Google if I cannot find
anything useful in DuckDuckGo.</p>
<p>So, here is what I did. I thought I would document it.</p>
<p>I have this Firefox plugin called <a href="https://addons.mozilla.org/en-US/firefox/addon/codeinjector/">Code Injector</a>.
What the plugin does is pretty simple, it lets me inject arbitrary html, css or js into any page.
I decided to just add an anchor tag <code>useGoogle()</code> (I was using <a href="https://reactjs.org/docs/hooks-overview.html">React hooks</a> extensively during that time)
under the DuckDuckGo search bar.</p>
<p>The end result will look something like this:</p>
<p><img src="https://blog.meain.io/img/ddg.png" alt="screenshot" /></p>
<p>If you click on <code>useGoogle()</code>, it will just redirect you to a page with the exact search on Google.
If you know some js, and css, it should be pretty simple.</p>
<p>But here is what I am using.</p>
<pre class="language-js"><code class="language-js">he <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">getElementsByClassName</span><span class="token punctuation">(</span><span class="token string">"header__search-wrap"</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br />sb <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"a"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />tex <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createTextNode</span><span class="token punctuation">(</span><span class="token string">"useGoogle()"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />sb<span class="token punctuation">.</span><span class="token function">setAttribute</span><span class="token punctuation">(</span><span class="token string">'style'</span><span class="token punctuation">,</span> <span class="token string">'margin-left:10px; color: #ccc;cursor:pointer;'</span><span class="token punctuation">)</span><br />sb<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>tex<span class="token punctuation">)</span><span class="token punctuation">;</span><br />he<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>sb<span class="token punctuation">)</span><span class="token punctuation">;</span><br />sb<span class="token punctuation">.</span><span class="token function-variable function">onclick</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> searchTerm <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">getElementById</span><span class="token punctuation">(</span><span class="token string">"search_form_input"</span><span class="token punctuation">)</span><span class="token punctuation">.</span>value<span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>searchTerm<span class="token punctuation">)</span><br /> window<span class="token punctuation">.</span>location <span class="token operator">=</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">https://www.google.com/search?q=</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>searchTerm<span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre>
<p><em>The variable names and stuff could be better, but I was in a hurry the day I set it up</em></p>
<blockquote>
<p>This combined with vim mode for Firefox lets me switch to DuckDuckGo even just with my keyboard(or you can always tab
over to the link).</p>
</blockquote>
<h1 id="directly-search-on-google%2C-youtube-or-github" tabindex="-1">Directly search on Google, YouTube or Github <a class="direct-link" href="https://blog.meain.io/2019/switching-to-duckduckgo/#directly-search-on-google%2C-youtube-or-github">#</a></h1>
<p>Well, in some cases you know that only Google will have to result, or you directly wanna search on YouTube or Github.</p>
<p>This is where you can use <a href="https://support.mozilla.org/en-US/kb/assign-shortcuts-search-engines">search shortcuts</a>.
I have my YouTube search assigned to <code>y</code>. So if I wanna search for something directly on YouTube, I just prepend <code>y </code>
in front of my search. You could even add one for Google.</p>
<p>That is all I got for now. I have been moving away from Google bit by bit, this was one of my first steps.</p>
Dynamic reverse proxy using nginx in Kubernetes2020-03-30T00:00:00Zhttps://blog.meain.io/2020/dynamic-reverse-proxy-kubernetes/<p>OK, first of all, let me make sure that you understand what we are trying to do here.</p>
<p>Let us say that I have a lot of <code>kubernetes</code> services with names like below. This list may grow or shrink dynamically
and is controlled by some other script.</p>
<p><em>I am spinning up a dev instances for each use as they login. Something new I am working on.</em></p>
<pre><code>exper-0
exper-1
exper-2
exper-3
exper-4
</code></pre>
<p>Now if I do not wanna create separate ingress for each item, how would I go about doing this?</p>
<blockquote>
<p>Heads Up. You cannot use default <code>nginx ingress</code> for this.</p>
</blockquote>
<h1 id="architecture" tabindex="-1">Architecture <a class="direct-link" href="https://blog.meain.io/2020/dynamic-reverse-proxy-kubernetes/#architecture">#</a></h1>
<p>Here is an overview of how we do this.</p>
<ul>
<li>Create a custom nginx deployment with a specific <code>nginx.conf</code> injected using a <code>configmap</code>.</li>
<li>Create a service which will expose this custom nginx deployment</li>
<li>Define an ingress with <code>*.mydomain.com</code> mapped to this service</li>
</ul>
<p>With this setup we will have <code>exper-0.mydomain.com</code> pointing to <code>exper-0</code>.</p>
<h1 id="code" tabindex="-1">Code <a class="direct-link" href="https://blog.meain.io/2020/dynamic-reverse-proxy-kubernetes/#code">#</a></h1>
<p>Now, going about actually doing it.</p>
<h2 id="nginx.conf-and-configmap.yaml" tabindex="-1"><code>nginx.conf</code> and <code>configmap.yaml</code> <a class="direct-link" href="https://blog.meain.io/2020/dynamic-reverse-proxy-kubernetes/#nginx.conf-and-configmap.yaml">#</a></h2>
<p>As we discussed above we need to define a custom nginx conf. This part is expained here.
The essence of your nginx conf is:</p>
<pre class="language-nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">server</span></span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">listen</span> <span class="token number">80</span></span><span class="token punctuation">;</span><br /><br /> <span class="token directive"><span class="token keyword">server_name</span> ~^(?<subdomain>.*?)\.</span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">resolver</span> kube-dns.kube-system.svc.cluster.local valid=5s</span><span class="token punctuation">;</span><br /><br /> <span class="token directive"><span class="token keyword">location</span> /</span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">proxy_pass</span> http://<span class="token variable">$subdomain</span>.mynamespace.svc.cluster.local</span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">proxy_set_header</span> Host <span class="token variable">$host</span></span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p><em>Replace <code>mynamespace</code> with the namespace you have this thing in</em></p>
<p>Let me explain what is going on.</p>
<ul>
<li>You listen on port 80. Basic.</li>
<li>For servername, you have a regex that will pull the first block of hostname to <code>subdomain</code>. This gets used a few
lines down.</li>
<li>You have to specify a resolver as nginx has to resolve this into and actual IP(internal IP of svc).</li>
<li>Now, inside location block you grab everything and pass it over to <code>http://$subdomain.mynamespace.svc.cluster.local</code> which will
resolve to the IP of the service.</li>
<li>You also need to change <code>Host</code> as otherwise it will be set as <code>http://$subdomain.mynamespace.svc.cluster.local</code> instead
of <code>exper-0.mydomain.com</code>.</li>
</ul>
<p>We actually need a bit more sutff to support websocket.</p>
<pre class="language-nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">proxy_set_header</span> Upgrade <span class="token variable">$http_upgrade</span></span><span class="token punctuation">;</span><br /><span class="token directive"><span class="token keyword">proxy_set_header</span> Connection <span class="token string">"Upgrade"</span></span><span class="token punctuation">;</span></code></pre>
<p>Add in a healthcheck.</p>
<pre class="language-nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">location</span> /healthz</span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">return</span> <span class="token number">200</span></span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>With all that in, it will look something like this.</p>
<pre class="language-nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">server</span></span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">listen</span> <span class="token number">80</span></span><span class="token punctuation">;</span><br /><br /> <span class="token directive"><span class="token keyword">server_name</span> ~^(?<subdomain>.*?)\.</span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">resolver</span> kube-dns.kube-system.svc.cluster.local valid=5s</span><span class="token punctuation">;</span><br /><br /> <span class="token directive"><span class="token keyword">location</span> /healthz</span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">return</span> <span class="token number">200</span></span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token directive"><span class="token keyword">location</span> /</span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">proxy_set_header</span> Upgrade <span class="token variable">$http_upgrade</span></span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">proxy_set_header</span> Connection <span class="token string">"Upgrade"</span></span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">proxy_pass</span> http://<span class="token variable">$subdomain</span>.msce0.svc.cluster.local</span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">proxy_set_header</span> Host <span class="token variable">$host</span></span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>We will also have to add some root config as we have to write a complete <code>nginx.conf</code>.
Pull all that together and drop it into a <code>configmap.yaml</code> and we have our first <code>kubernetes</code> object.</p>
<p><strong>configmap.yaml</strong></p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> ConfigMap<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> confnginx<br /><span class="token key atrule">data</span><span class="token punctuation">:</span><br /> <span class="token key atrule">nginx.conf</span><span class="token punctuation">:</span> <span class="token punctuation">|</span><span class="token scalar string"><br /> user nginx;<br /> worker_processes 1;<br /> error_log /var/log/nginx/error.log warn;<br /> pid /var/run/nginx.pid;<br /> events {<br /> worker_connections 1024;<br /> }<br /> http {<br /> include /etc/nginx/mime.types;<br /> default_type application/octet-stream;<br /> log_format main '$remote_addr - $remote_user [$time_local] "$request" '<br /> '$status $body_bytes_sent "$http_referer" '<br /> '"$http_user_agent" "$http_x_forwarded_for"';<br /> access_log /var/log/nginx/access.log main;<br /> sendfile on;<br /> keepalive_timeout 65;<br /> server {<br /> listen 80;</span><br /><br /> server_name ~^(<span class="token punctuation">?</span><subdomain<span class="token punctuation">></span>.<span class="token important">*?)\.;</span><br /> resolver kube<span class="token punctuation">-</span>dns.kube<span class="token punctuation">-</span>system.svc.cluster.local valid=5s;<br /><br /> location /healthz <span class="token punctuation">{</span><br /> return 200;<br /> <span class="token punctuation">}</span><br /><br /> location / <span class="token punctuation">{</span><br /> proxy_set_header Upgrade $http_upgrade;<br /> proxy_set_header Connection "Upgrade";<br /> proxy_pass http<span class="token punctuation">:</span>//$subdomain.msce0.svc.cluster.local;<br /> proxy_set_header Host $host;<br /> proxy_http_version 1.1;<br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span></code></pre>
<h2 id="deployment.yaml-and-service.yaml" tabindex="-1"><code>deployment.yaml</code> and <code>service.yaml</code> <a class="direct-link" href="https://blog.meain.io/2020/dynamic-reverse-proxy-kubernetes/#deployment.yaml-and-service.yaml">#</a></h2>
<p>There is nothing fancy in here. Just a plain old <code>deployment.yaml</code> and <code>service.yaml</code>.</p>
<blockquote>
<p>Inside <code>deployment.yaml</code> we have to make sure to load the <code>configmap</code> we setup. See the <code>volumeMounts</code> section.</p>
</blockquote>
<p><strong>deployment.yaml</strong></p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> apps/v1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> Deployment<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> nginx<br /> <span class="token key atrule">labels</span><span class="token punctuation">:</span><br /> <span class="token key atrule">app</span><span class="token punctuation">:</span> nginx<br /><span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">selector</span><span class="token punctuation">:</span><br /> <span class="token key atrule">matchLabels</span><span class="token punctuation">:</span><br /> <span class="token key atrule">app</span><span class="token punctuation">:</span> nginx<br /> <span class="token key atrule">replicas</span><span class="token punctuation">:</span> <span class="token number">1</span><br /> <span class="token key atrule">template</span><span class="token punctuation">:</span><br /> <span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">labels</span><span class="token punctuation">:</span><br /> <span class="token key atrule">app</span><span class="token punctuation">:</span> nginx<br /> <span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">containers</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> nginx<br /> <span class="token key atrule">image</span><span class="token punctuation">:</span> nginx<span class="token punctuation">:</span>alpine<br /> <span class="token key atrule">ports</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">containerPort</span><span class="token punctuation">:</span> <span class="token number">80</span><br /> <span class="token key atrule">volumeMounts</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> nginx<span class="token punctuation">-</span>config<br /> <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /etc/nginx/nginx.conf<br /> <span class="token key atrule">subPath</span><span class="token punctuation">:</span> nginx.conf<br /> <span class="token key atrule">volumes</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> nginx<span class="token punctuation">-</span>config<br /> <span class="token key atrule">configMap</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> confnginx</code></pre>
<p><strong>service.yaml</strong></p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">kind</span><span class="token punctuation">:</span> Service<br /><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> nginx<span class="token punctuation">-</span>custom<br /><span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">selector</span><span class="token punctuation">:</span><br /> <span class="token key atrule">app</span><span class="token punctuation">:</span> nginx<br /> <span class="token key atrule">ports</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">protocol</span><span class="token punctuation">:</span> TCP<br /> <span class="token key atrule">port</span><span class="token punctuation">:</span> <span class="token number">80</span><br /> <span class="token key atrule">targetPort</span><span class="token punctuation">:</span> <span class="token number">80</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> nginx</code></pre>
<h2 id="ingress.yaml" tabindex="-1"><code>ingress.yaml</code> <a class="direct-link" href="https://blog.meain.io/2020/dynamic-reverse-proxy-kubernetes/#ingress.yaml">#</a></h2>
<p>Now with the service setup, we can use an ingress to give it a <code>hostname</code>.</p>
<p><strong>ingress.yaml</strong></p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> networking.k8s.io/v1beta1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> Ingress<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> ingress<span class="token punctuation">-</span>nginx<span class="token punctuation">-</span>custom<br /> <span class="token key atrule">annotations</span><span class="token punctuation">:</span><br /> <span class="token key atrule">kubernetes.io/ingress.class</span><span class="token punctuation">:</span> nginx<br /><span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">rules</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">host</span><span class="token punctuation">:</span> <span class="token string">'*.mydomain.com'</span><br /> <span class="token key atrule">http</span><span class="token punctuation">:</span><br /> <span class="token key atrule">paths</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">path</span><span class="token punctuation">:</span> /<br /> <span class="token key atrule">backend</span><span class="token punctuation">:</span><br /> <span class="token key atrule">serviceName</span><span class="token punctuation">:</span> nginx<span class="token punctuation">-</span>custom<br /> <span class="token key atrule">servicePort</span><span class="token punctuation">:</span> <span class="token number">80</span></code></pre>
<h1 id="deploy" tabindex="-1">Deploy <a class="direct-link" href="https://blog.meain.io/2020/dynamic-reverse-proxy-kubernetes/#deploy">#</a></h1>
<p>Well, I am pretty sure you know how to do this. But essentially</p>
<pre class="language-shell"><code class="language-shell">k apply <span class="token parameter variable">-f</span> configmap.yaml<br />k apply <span class="token parameter variable">-f</span> deployment.yaml<br />k apply <span class="token parameter variable">-f</span> service.yaml<br />k apply <span class="token parameter variable">-f</span> ingress.yaml</code></pre>
<center><h1>K. THX. BYE</h1></center>
Auto shutdown VM if no active SSH connections2020-04-05T00:00:00Zhttps://blog.meain.io/2020/auto-shutdown-no-ssh/<p>Here is a simple workflow that I have extracted out into a blog.
I generally have a VM on standby to offload some tasks over to that VM instead of running things on my laptop.</p>
<p>One main example of this is when I am trying to do some experiment and check if a container still builds after I do some
changes. I usually make the changes and offload the build piece over to the VM. A good thing about VM is that it is more
powerful that my computer, has more storage for more caching, and also a much faster internet connection.
<em>It was an obvious choice.</em></p>
<p>I have a script which will rsync files into the VM and run a docker build. <a href="https://github.com/meain/dotfiles/blob/64c9fcb00d0ce0ef3af96426cda84db5cd64f6e8/scripts/.local/bin/utils/%2Ctry-build">Here</a>
is the script available on my Github.</p>
<p>I don't need it to on when not in use. I am nice person and usually shuts it down after I need it but the thing is I at
times forget to do so. That is when I decided that I could probably write something to check if I am using that and shut
down if I am not.</p>
<p>The solution that I came up with was to check if there has been any SSH connections in the past hour and shut down
the VM if none. Here is the script that I am using.</p>
<pre class="language-shell"><code class="language-shell"><span class="token keyword">if</span> last <span class="token operator">|</span> <span class="token function">grep</span> <span class="token string">"still logged in"</span><span class="token punctuation">;</span><span class="token keyword">then</span><br /> <span class="token builtin class-name">exit</span> <span class="token number">0</span><br /><span class="token keyword">fi</span><br /><br /><span class="token assign-left variable">LAST_ACCESS</span><span class="token operator">=</span><span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token function">stat</span> -c<span class="token string">'%X'</span> /var/log/wtmp<span class="token variable">)</span></span>"</span><br /><span class="token assign-left variable">CURRENT_TIME</span><span class="token operator">=</span><span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token function">date</span> +%s<span class="token variable">)</span></span>"</span><br /><span class="token assign-left variable">DIFF</span><span class="token operator">=</span><span class="token string">"<span class="token variable"><span class="token variable">$((</span>CURRENT_TIME<span class="token operator">-</span>LAST_ACCESS<span class="token variable">))</span></span>"</span><br /><span class="token keyword">if</span> <span class="token punctuation">[</span> <span class="token variable">$DIFF</span> <span class="token parameter variable">-ge</span> <span class="token number">3600</span> <span class="token punctuation">]</span><span class="token punctuation">;</span><span class="token keyword">then</span><br /> <span class="token function">sudo</span> <span class="token function">shutdown</span><br /><span class="token keyword">fi</span></code></pre>
<p>Let me explain.</p>
<p>First we check if there are any active SSH connections to the machine. We can do that by using the <code>last</code> command.
Here is the output of that command.</p>
<p><img src="https://blog.meain.io/img/ssh-last.png" alt="screenshot" /></p>
<p>The first two lines of the script are for this. It checks if the string "still logged in" is in the output of the <code>last</code>
command. If so, we know that there is an active ssh connection and we can abort the script.</p>
<p>If not, then we have to check when the last connection was closed.
There are a lot of commands that will give you the complete login details as to when someone logged in and out.
But the simplest route for us is to check when the file <code>/var/log/wtmp</code> was last written.</p>
<p>This is a file which contains info about ssh connections, so the last time this got updated will be the last time
someone logged out or in to the machine. And since we checked that there are no active sessions, this will be last time
someone logged out.</p>
<p>To get the last accessed time on <code>/var/log/wtmp</code> we can use the <code>stat</code> command.</p>
<p><img src="https://blog.meain.io/img/ssh-stat.png" alt="screenshot" /></p>
<p>We can compare this time to the current time and check if the difference is more that 60*60 (1hour). You can vary this
time with what you feel comfortable.
If the check return true, we just do a shutdown.</p>
<p>Now with this script placed somewhere we can just call this script from a cronjob.
And now you have a VM that will shut itself down after use.</p>
<p>One possible improvement is that we could probably looks for processes which might be running with high cpu load and
not shutdown if that is the case. But I always have an ssh connection if I am working, so this works for me.</p>
Mounting S3 bucket in docker containers on kubernetes2020-04-12T00:00:00Zhttps://blog.meain.io/2020/mounting-s3-bucket-kube/<p>Another installment of me figuring out more of kubernetes.</p>
<blockquote>
<p>Full code available at <a href="https://github.com/meain/s3-mounter">meain/s3-mounter</a></p>
</blockquote>
<p>So, I was working on a project which will let people login to a web service and spin up a coding env with prepopulated
data and creds. We were spinning up kube pods for each user.</p>
<p>All of our data is in s3 buckets, so it would have been really easy if could just mount s3 buckets in the docker
container. My initial thought was that there would be some <a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/">PV</a> which I could use, but it can't be that simple right.</p>
<p>So after some hunting, I thought I would just mount the s3 bucket as a volume in the pod. I was not sure if this was the
right way to go, but I thought I would go with this anyways.</p>
<p>I found this repo <a href="https://github.com/s3fs-fuse/s3fs-fuse">s3fs-fuse/s3fs-fuse</a> which will let you mount s3.
Tried it out in my local and it seemed to work pretty well. Could not get it to work in a docker container initially but
figured out that I just had to give the container extra privileges.</p>
<p>Adding <code>--privileged</code> to the docker command takes care of that.</p>
<blockquote>
<p>UPDATE (<time datetime="2023-03-27">Mar 27 2023</time>):
<a href="https://github.com/awslabs/mountpoint-s3">mountpoint</a> (still in
alpha) is an official alternative to create a mount from s3
bucket. This can be used instead of <code>s3fs</code> mentioned in the blog.</p>
</blockquote>
<p>Now to actually get it running on k8s.</p>
<h3 id="step-1%3A-create-docker-image" tabindex="-1">Step 1: Create Docker image <a class="direct-link" href="https://blog.meain.io/2020/mounting-s3-bucket-kube/#step-1%3A-create-docker-image">#</a></h3>
<p>This was relatively straight foreward, all I needed to do was to pull an alpine image and installing
<a href="https://github.com/s3fs-fuse/s3fs-fuse">s3fs-fuse/s3fs-fuse</a> on to it.</p>
<p>Just build the following container and push it to your container.
I have published this image on my Dockerhub. You can use <a href="https://hub.docker.com/r/meain/s3-mounter">that</a> if you want.</p>
<pre class="language-docker"><code class="language-docker"><span class="token instruction"><span class="token keyword">FROM</span> alpine:3.3</span><br /><br /><span class="token instruction"><span class="token keyword">ENV</span> MNT_POINT /var/s3fs</span><br /><span class="token instruction"><span class="token keyword">ARG</span> S3FS_VERSION=v1.86</span><br /><br /><span class="token instruction"><span class="token keyword">RUN</span> apk --update --no-cache add fuse alpine-sdk automake autoconf libxml2-dev fuse-dev curl-dev git bash; <span class="token operator">\</span><br /> git clone https://github.com/s3fs-fuse/s3fs-fuse.git; <span class="token operator">\</span><br /> cd s3fs-fuse; <span class="token operator">\</span><br /> git checkout tags/<span class="token variable">${S3FS_VERSION}</span>; <span class="token operator">\</span><br /> ./autogen.sh; <span class="token operator">\</span><br /> ./configure --prefix=/usr; <span class="token operator">\</span><br /> make; <span class="token operator">\</span><br /> make install; <span class="token operator">\</span><br /> make clean; <span class="token operator">\</span><br /> rm -rf /var/cache/apk/*; <span class="token operator">\</span><br /> apk del git automake autoconf;</span><br /><br /><span class="token instruction"><span class="token keyword">RUN</span> mkdir -p <span class="token string">"$MNT_POINT"</span></span><br /><br /><span class="token instruction"><span class="token keyword">COPY</span> run.sh run.sh</span><br /><span class="token instruction"><span class="token keyword">CMD</span> ./run.sh</span></code></pre>
<p><strong><a href="http://run.sh/">run.sh</a></strong></p>
<pre class="language-shell"><code class="language-shell"><span class="token builtin class-name">echo</span> <span class="token string">"<span class="token variable">$AWS_KEY</span>:<span class="token variable">$AWS_SECRET_KEY</span>"</span> <span class="token operator">></span> <span class="token function">passwd</span> <span class="token operator">&&</span> <span class="token function">chmod</span> <span class="token number">600</span> <span class="token function">passwd</span><br />s3fs <span class="token string">"<span class="token variable">$S3_BUCKET</span>"</span> <span class="token string">"<span class="token variable">$MNT_POINT</span>"</span> <span class="token parameter variable">-o</span> <span class="token assign-left variable">passwd_file</span><span class="token operator">=</span>passwd <span class="token operator">&&</span> <span class="token function">tail</span> <span class="token parameter variable">-f</span> /dev/null</code></pre>
<h3 id="step-2%3A-create-a-secret" tabindex="-1">Step 2: Create a Secret <a class="direct-link" href="https://blog.meain.io/2020/mounting-s3-bucket-kube/#step-2%3A-create-a-secret">#</a></h3>
<p>The Dockerfile does not really contain any specific items like bucket name or key. Here we use a <code>Secret</code> to inject
values into the docker container. A sample <code>Secret</code> will look something like this.<br />
Replace the empty values with your specific data.
Afer that just <code>k apply -f secret.yaml</code></p>
<p><strong>secret.yaml</strong></p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> Secret<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> s3<span class="token punctuation">-</span>credentials<br /><span class="token key atrule">data</span><span class="token punctuation">:</span><br /> <span class="token key atrule">S3_REGION</span><span class="token punctuation">:</span> <span class="token string">""</span><br /> <span class="token key atrule">S3_BUCKET</span><span class="token punctuation">:</span> <span class="token string">""</span><br /> <span class="token key atrule">AWS_KEY</span><span class="token punctuation">:</span> <span class="token string">""</span><br /> <span class="token key atrule">AWS_SECRET_KEY</span><span class="token punctuation">:</span> <span class="token string">""</span></code></pre>
<h3 id="step-3%3A-create-daemonset" tabindex="-1">Step 3: Create DaemonSet <a class="direct-link" href="https://blog.meain.io/2020/mounting-s3-bucket-kube/#step-3%3A-create-daemonset">#</a></h3>
<p>Well we could technically just have this mounting in each container, but this is a better way to go.
What we are doing is that we mount s3 to the container but the folder that we mount to, is mapped to host machine.</p>
<p>With this, we will easily be able to get the folder from the host machine in any other container just as if we are
mounting a normal fs.</p>
<p>The visualisation from <a href="https://github.com/freegroup/kube-s3">freegroup/kube-s3</a> makes it pretty clear.</p>
<p><img src="https://blog.meain.io/img/s3-mount.png" alt="screenshot" /></p>
<p>Since every pod expects the item to be available in the host fs, we need to make sure all host VMs do have the folder. A
<code>DaemonSet</code> will let us do that. A <code>DaemonSet</code> pretty much ensures that one of this container will be run on every node
which you specify. In our case, we ask it to run on all nodes.</p>
<p>Once ready <code>k apply -f daemonset.yaml</code>.</p>
<blockquote>
<p>If you check the file, you can see that we are mapping /var/s3fs to /mnt/s3data on host</p>
</blockquote>
<p><strong>daemonset.yaml</strong></p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> extensions/v1beta1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> DaemonSet<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">labels</span><span class="token punctuation">:</span><br /> <span class="token key atrule">app</span><span class="token punctuation">:</span> s3<span class="token punctuation">-</span>provider<br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> s3<span class="token punctuation">-</span>provider<br /><span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">template</span><span class="token punctuation">:</span><br /> <span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">labels</span><span class="token punctuation">:</span><br /> <span class="token key atrule">app</span><span class="token punctuation">:</span> s3<span class="token punctuation">-</span>provider<br /> <span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">containers</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> s3fuse<br /> <span class="token key atrule">image</span><span class="token punctuation">:</span> meain/s3<span class="token punctuation">-</span>mounter<br /> <span class="token key atrule">securityContext</span><span class="token punctuation">:</span><br /> <span class="token key atrule">privileged</span><span class="token punctuation">:</span> <span class="token boolean important">true</span><br /> <span class="token key atrule">envFrom</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">secretRef</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> s3<span class="token punctuation">-</span>credentials<br /> <span class="token key atrule">volumeMounts</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> devfuse<br /> <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /dev/fuse<br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> mntdatas3fs<br /> <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /var/s3fs<span class="token punctuation">:</span>shared<br /> <span class="token key atrule">volumes</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> devfuse<br /> <span class="token key atrule">hostPath</span><span class="token punctuation">:</span><br /> <span class="token key atrule">path</span><span class="token punctuation">:</span> /dev/fuse<br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> mntdatas3fs<br /> <span class="token key atrule">hostPath</span><span class="token punctuation">:</span><br /> <span class="token key atrule">path</span><span class="token punctuation">:</span> /mnt/s3data</code></pre>
<blockquote>
<p>If you are using GKE and using <a href="https://cloud.google.com/container-optimized-os/docs/concepts/disks-and-filesystem">Container-Optimized OS</a>,
<code>/mnt</code> will not be writeable, use <code>/home/s3data</code> instead</p>
</blockquote>
<p>With that applied you will have :</p>
<pre><code>$ k get all
NAME READY STATUS RESTARTS AGE
pod/s3-provider-psp9v 1/1 Running 0 39m
pod/s3-provider-zvfrs 1/1 Running 0 39m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/s3-provider 2 2 2 2 2 <none> 39m
</code></pre>
<p>By now, you should have the host system with s3 mounted on <code>/mnt/s3data</code>.
You can check that by running the command <code>k exec -it s3-provider-psp9v -- ls /var/s3fs</code></p>
<h3 id="step-4%3A-running-your-actual-container" tabindex="-1">Step 4: Running your actual container <a class="direct-link" href="https://blog.meain.io/2020/mounting-s3-bucket-kube/#step-4%3A-running-your-actual-container">#</a></h3>
<p>With all that setup, now you are ready to go in and actually do what you started out to do. I will show a really simple
pod spec.</p>
<p><strong>pod.yaml</strong></p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> Pod<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> test<span class="token punctuation">-</span>pd<br /><span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">containers</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">image</span><span class="token punctuation">:</span> nginx<br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> s3<span class="token punctuation">-</span>test<span class="token punctuation">-</span>container<br /> <span class="token key atrule">securityContext</span><span class="token punctuation">:</span><br /> <span class="token key atrule">privileged</span><span class="token punctuation">:</span> <span class="token boolean important">true</span><br /> <span class="token key atrule">volumeMounts</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> mntdatas3fs<br /> <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /var/s3fs<span class="token punctuation">:</span>shared<br /> <span class="token key atrule">volumes</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> mntdatas3fs<br /> <span class="token key atrule">hostPath</span><span class="token punctuation">:</span><br /> <span class="token key atrule">path</span><span class="token punctuation">:</span> /mnt/s3data</code></pre>
<p>Run this and if you check in <code>/var/s3fs</code>, you can see the same files you have in your s3 bucket.</p>
<p><em>Change <code>mountPath</code> to change where it gets mounted to. Change <code>hostPath.path</code> to a subdir if you only want to expose on
specific folder</em></p>
<h2 id="extra-resources" tabindex="-1">Extra resources <a class="direct-link" href="https://blog.meain.io/2020/mounting-s3-bucket-kube/#extra-resources">#</a></h2>
<ul>
<li>
<p><a href="https://github.com/freegroup/kube-s3">freegroup/kube-s3</a></p>
</li>
<li>
<p><a href="https://github.com/s3fs-fuse/s3fs-fuse">s3fs-fuse/s3fs-fuse</a></p>
</li>
<li>
<p><a href="https://github.com/skypeter1/docker-s3-bucket">skypeter1/docker-s3-bucket</a></p>
</li>
<li>
<p><a href="https://icicimov.github.io/blog/virtualization/Kubernetes-shared-storage-with-S3-backend/">Kubernetes-shared-storage-with-S3-backend</a></p>
</li>
</ul>
A simpler method for a reactive UI2020-05-16T00:00:00Zhttps://blog.meain.io/2020/a-simpler-method-for-reactive-ui/<p>Hola¡</p>
<p>I have been doing frontend for a long time. I started off with vanilla JS, then used jQuery and after that moved
directly onto using pretty much React. I have tried out a lot of others and most of them seem to have this idea of
having the UI as a pure function of your data at the core.</p>
<p>I really liked the idea of having your UI as a function of your data. This idea really stuck with me that I loved using
React for just that reason alone that I started using React in really tiny projects. But started moving away from it
using this.</p>
<p><em>Also, unpopular opinion, I do like JSX. It lets me write my HTML in my JS instead of the wrong way.</em></p>
<p>This is nothing new, just a workflow that works really well for me. You might wanna still reach out for libraries
when you are building something big. This works best for small side projects.</p>
<p>As you might expect, there are two main peices. The data and the rendering logic.</p>
<p>Your data lies in a global dict, to which you get and set using a function. You could probably even set up getters and
setters to each key in the dict for what we are doing here to work. This is what Vue does instead of React's <code>setState()</code>.</p>
<p>A sample signature of your <code>update()</code> function will look like this:</p>
<pre class="language-javascript"><code class="language-javascript"><span class="token function">update</span><span class="token punctuation">(</span><span class="token string">"key.name"</span><span class="token punctuation">,</span> object<span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>So the idea is that, you set your object using a key. You can go down to a child using a <code>.</code>. Well, this is not the most
optimal since you loose the ability to have dots in your object keys, but I don't really care about that.</p>
<p>Now for rendering this global object that you are setting , it can have different callbacks assigned to each key.
You could technically call this a library, but here is the gist of what you will need.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="token keyword">const</span> global_object <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token keyword">const</span> callbacks <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">function</span> <span class="token function">process_callbacks</span><span class="token punctuation">(</span><span class="token parameter">key<span class="token punctuation">,</span> value</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>key <span class="token keyword">in</span> callbacks<span class="token punctuation">)</span> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">let</span> callback <span class="token keyword">of</span> callbacks<span class="token punctuation">[</span>key<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token function">callback</span><span class="token punctuation">(</span>value<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">export</span> <span class="token keyword">function</span> <span class="token function">register</span><span class="token punctuation">(</span><span class="token parameter">key<span class="token punctuation">,</span> callback</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>key <span class="token keyword">in</span> callbacks<span class="token punctuation">)</span> callbacks<span class="token punctuation">[</span>key<span class="token punctuation">]</span><span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span>callback<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">else</span> callbacks<span class="token punctuation">[</span>key<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token punctuation">[</span>callback<span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token function">callback</span><span class="token punctuation">(</span><span class="token function">get</span><span class="token punctuation">(</span>key<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">export</span> <span class="token keyword">function</span> <span class="token function">update</span><span class="token punctuation">(</span><span class="token parameter">key<span class="token punctuation">,</span> value</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token comment">// handle nested keys</span><br /> global_object<span class="token punctuation">[</span>key<span class="token punctuation">]</span> <span class="token operator">=</span> value<span class="token punctuation">;</span><br /> <span class="token function">process_callbacks</span><span class="token punctuation">(</span>key<span class="token punctuation">,</span> value<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">export</span> <span class="token keyword">function</span> <span class="token function">get</span><span class="token punctuation">(</span><span class="token parameter">key</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token comment">// handle nested keys</span><br /> <span class="token keyword">return</span> global_object<span class="token punctuation">[</span>key<span class="token punctuation">]</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>So, each and every render function you write, you can attach a callback on the global callback dict.</p>
<p>Well, now to writing our templates. You can actually just write it with just <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals">template
literals</a>, but you might wanna look
into <a href="https://lit-html.polymer-project.org/">lit-html</a>.</p>
<h2 id="sample-code" tabindex="-1">Sample code <a class="direct-link" href="https://blog.meain.io/2020/a-simpler-method-for-reactive-ui/#sample-code">#</a></h2>
<p><a href="https://codesandbox.io/s/exciting-nobel-n0z9v?file=/src/index.js">Here</a> is a CodeSandbox link to a hello world clicker demo,
and <a href="https://codesandbox.io/s/solitary-violet-1fy4z?file=/src/index.js">here</a> is a lit-html version.</p>
<pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>div</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>button</span> <span class="token attr-name">id</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>clicker<span class="token punctuation">"</span></span><span class="token punctuation">></span></span>Click me<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>button</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>div</span><span class="token punctuation">></span></span></code></pre>
<pre class="language-javascript"><code class="language-javascript"><span class="token keyword">const</span> clicker <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">getElementById</span><span class="token punctuation">(</span><span class="token string">"clicker"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />clicker<span class="token punctuation">.</span><span class="token function-variable function">onclick</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token function">update</span><span class="token punctuation">(</span><span class="token string">"counter"</span><span class="token punctuation">,</span> <span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"counter"</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token function">register</span><span class="token punctuation">(</span><span class="token string">"counter"</span><span class="token punctuation">,</span> <span class="token parameter">value</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> clicker<span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string"><span></span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>value <span class="token operator">===</span> <span class="token number">0</span> <span class="token operator">?</span> <span class="token string">"Click me"</span> <span class="token operator">:</span> value<span class="token interpolation-punctuation punctuation">}</span></span><span class="token string"></span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
What exactly are stock options?2020-05-20T00:00:00Zhttps://blog.meain.io/2020/what-exactly-are-stock-options/<p>So yeah, this is me trying to write a non technical blog. Not sure how this will end up.
This is just an intro, very basics of what I wished I knew when I started off.</p>
<p>I am not someone who really knows what I am talking about here, you have been warned.</p>
<h1 id="what-are-they%3F" tabindex="-1">What are they? <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#what-are-they%3F">#</a></h1>
<p>Stock options are an agreement that the company that you work for gives you about its stocks.
The agreement pretty much says that you can buy a certain amount of their stocks at a fixed price, which usually is the
current stock price of that company or the <a href="https://www.investopedia.com/terms/f/fairvalue.asp">fair valuation</a> price
in case the company has not gone public.</p>
<p>In the case of this fair valuation thingy, usually an external agency does a review on the company and gives a figure as
to what the company stock price would be in case the company had gone public.</p>
<h1 id="why-are-they%3F" tabindex="-1">Why are they? <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#why-are-they%3F">#</a></h1>
<p>So yeah, you get this thing called salary when you work for a company.
But along with that at times you get these things as well.</p>
<h2 id="why-would-you-want-it%3F" tabindex="-1">Why would you want it? <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#why-would-you-want-it%3F">#</a></h2>
<p>Good question. If you really believe that the startup you are working for is actually gonna make it, your stocks (or
stock options) will be worth much more than the initial price that they were offered to you for. If you do not believe
in the company, why are you working there?</p>
<h2 id="why-would-the-company-give-it%3F" tabindex="-1">Why would the company give it? <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#why-would-the-company-give-it%3F">#</a></h2>
<p>A lot of people working there expects it. Plus, by giving employees stock options instead of a direct salary they have
more incentive to making sure the company ends up being successful. Also, maybe in the case of a promising initial stage
startup without a lot of funding this would be great option.</p>
<h1 id="sample-workflow" tabindex="-1">Sample workflow <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#sample-workflow">#</a></h1>
<h2 id="company-offers-you-stock-options" tabindex="-1">Company offers you stock options <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#company-offers-you-stock-options">#</a></h2>
<p>This is the initial step. The company essentially gives you the option to but some stocks now or in the future.
I believe this is called as an option grant. At this point you are not buying any stocks, or saying that you will.
You are just given an option to buy the stocks some time in the future at the said price. All you are doing when you
agree is that you are saying that the price and number of stocks works for you.</p>
<h2 id="buying-the-stocks" tabindex="-1">Buying the stocks <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#buying-the-stocks">#</a></h2>
<p>Now that you have your stocks options available and you are siting home thinking "hmm, maybe I should buy the stocks"
you get to this step. This is where you actually purchase the stocks that you were "granted" before. Even if the
stock price of that company has gone up, you can still buy it at the same price as the one that is mentioned in the
option grant.</p>
<h3 id="caveats" tabindex="-1">Caveats <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#caveats">#</a></h3>
<h4 id="vesting-schedule" tabindex="-1">Vesting schedule <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#vesting-schedule">#</a></h4>
<p>Just because you have accepted the grant does not mean that you can buy the stocks. Usually, there is a thing called a
<a href="https://www.upcounsel.com/vesting-schedule">vesting schedule</a>. Essentially the idea is that you cannot buy up all the
stocks and leave the company just because they gave you an option grant. This is nothing fancy, this just limits how
much stocks you can buy based on how much time you have worked for the company.</p>
<p>Vesting schedule can change from company to company. A sample vesting schedule can be something like, you can buy 25% of
the stocks after 1 year. After than you can buy a small chunk each month for the next 4 years.</p>
<h4 id="expiry" tabindex="-1">Expiry <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#expiry">#</a></h4>
<p>Also, these things(grants) expire. You cannot just hold them forever and buy whenever. There is usually a time period
from when you accepted the grant to when you the time till when you can buy those stocks.</p>
<p>A sample expiry could be like within 10 years or within 30 days from you leaving the company.</p>
<h2 id="once-you-buy" tabindex="-1">Once you buy <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#once-you-buy">#</a></h2>
<p>OK, you have made up your mind and decided to buy those stocks thingies that they gave you. Now what.
There are usually two situations.</p>
<h3 id="%231---company-is-public" tabindex="-1">#1 - Company is public <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#%231---company-is-public">#</a></h3>
<p>In the case that your company is already public, you can just sell that stocks at their current price to whomever is
willing to buy it.</p>
<h3 id="%232---company-is-not-public" tabindex="-1">#2 - Company is not public <a class="direct-link" href="https://blog.meain.io/2020/what-exactly-are-stock-options/#%232---company-is-not-public">#</a></h3>
<p>In this case, your stock selling options are limited, kinda. You can sell them to others, but you might require an NOC
from the company whose shares you are selling, to sell it to that person. There is also an option for the company to buy back
the stocks from you, not necessary that they have to.</p>
Completely remove a file from git history2020-05-22T00:00:00Zhttps://blog.meain.io/2020/completely-remove-a-file-from-git-history/<blockquote>
<p>UPDATE: I did run into some pretty good documentation from GitHub
<a href="https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository">here</a>. One
caveat is that inorder to remove it from pull requests, you will
have to get the GitHub team involved.</p>
</blockquote>
<p>Another short one.</p>
<p>Imagine this situation, you committed a file containing passwords into git accidentally.
What do you do?</p>
<p>Pretty easy, you just revert that commit.
The command below show do the trick.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">git</span> reset <span class="token parameter variable">--soft</span> HEAD^ <span class="token operator">&&</span> <span class="token function">git</span> reset</code></pre>
<p>But what if you have made multiple commits after doing this? Yeah, this is messier. But you can still do it.
Here is what the code will look like.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function-name function">git_prune_file</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">git</span> filter-branch <span class="token parameter variable">--force</span> --index-filter <span class="token punctuation">\</span><br /> <span class="token string">"git rm --cached --ignore-unmatch '<span class="token variable">$1</span>'"</span> <span class="token punctuation">\</span><br /> --prune-empty --tag-name-filter <span class="token function">cat</span> -- <span class="token parameter variable">--all</span><br /><span class="token punctuation">}</span></code></pre>
<p>With this in place, you can call <code>git_prune_file</code> on the file that you want to be removed from history.
This will go through all the commits and remove that file. Pretty sweet, right?</p>
<p>Just for fun, <a href="https://youtu.be/1ariej5xvfc">here</a> is it in action.</p>
Copy file opened in macOS preview to working directory2020-06-02T00:00:00Zhttps://blog.meain.io/2020/copy-last-opened-file-to-current-folder/<p>Another random script I pieced together recently.</p>
<p>I generally have a bunch of temp folders that I keep around for files to go in. And at times, I have pull some files
into the current folder I am working on in my shell. If those are pdf of image files, I usually have them open in macOS
preview which btw is a neat tool. Anyhow, when I now have to copy the file to my current location inside the terminal I
have to look up the folder, filename and God help us if there are spaces in the filename in which case I have to go back
and add quotes.</p>
<p>So I decided to just write a script to do just that. It looks up the file in the last opened preview window and copies
that file to my current directory and gives me an option to change the name just in case. Here is the script.</p>
<pre class="language-shell"><code class="language-shell"><span class="token assign-left variable">FILE</span><span class="token operator">=</span><span class="token string">"<span class="token variable"><span class="token variable">$(</span>osascript <span class="token parameter variable">-e</span> <span class="token string">'tell application "Preview" to return path of back document'</span><span class="token variable">)</span></span>"</span><br /><span class="token builtin class-name">printf</span> <span class="token string">"New filename(%s): "</span> <span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token function">basename</span> <span class="token string">"<span class="token variable">$FILE</span>"</span><span class="token variable">)</span></span>"</span><br /><span class="token builtin class-name">read</span> <span class="token parameter variable">-r</span><br /><span class="token punctuation">[</span> <span class="token parameter variable">-n</span> <span class="token string">"<span class="token environment constant">$REPLY</span>"</span> <span class="token punctuation">]</span> <span class="token operator">&&</span> <span class="token assign-left variable">NEW</span><span class="token operator">=</span><span class="token string">"<span class="token environment constant">$REPLY</span>"</span> <span class="token operator">||</span> <span class="token assign-left variable">NEW</span><span class="token operator">=</span><span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token function">basename</span> <span class="token string">"<span class="token variable">$FILE</span>"</span><span class="token variable">)</span></span>"</span><br /><span class="token function">cp</span> <span class="token string">"<span class="token variable">$FILE</span>"</span> <span class="token string">"<span class="token variable">$NEW</span>"</span></code></pre>
<h3 id="explanation" tabindex="-1">Explanation <a class="direct-link" href="https://blog.meain.io/2020/copy-last-opened-file-to-current-folder/#explanation">#</a></h3>
<p>First we get the name of the file using <code>applescript</code>. This is the important part.</p>
<pre class="language-applescript"><code class="language-applescript"><span class="token keyword">tell</span> <span class="token class-name">application</span> <span class="token string">"Preview"</span> <span class="token keyword">to</span> <span class="token keyword">return</span> path <span class="token keyword">of</span> <span class="token keyword">back</span> document</code></pre>
<p>Once we have that we just ask for a new name and then copy the file from current location to here.
<code>basename</code> lets use get just the filename from a file path.</p>
<h3 id="extra" tabindex="-1">Extra <a class="direct-link" href="https://blog.meain.io/2020/copy-last-opened-file-to-current-folder/#extra">#</a></h3>
<p>Usually you could also make use of the directory of the last <code>Finder</code> window. Like, cd into last open <code>Finder</code> location.
This is how you would go about getting the folder location.</p>
<pre class="language-applescript"><code class="language-applescript"><span class="token keyword">tell</span> <span class="token class-name">application</span> <span class="token string">"Finder"</span> <span class="token keyword">to</span> <span class="token keyword">set</span> currentDir <span class="token keyword">to</span> target <span class="token keyword">of</span> Finder window <span class="token number">1</span> <span class="token operator">as</span> <span class="token class-name">alias</span><br />log POSIX path <span class="token keyword">of</span> currentDir</code></pre>
<p>This will return you the path of the topmost Finder window. If no finder window is open it will just error out.</p>
What you need might not be Kubernetes2020-06-21T00:00:00Zhttps://blog.meain.io/2020/you-might-not-need-kubernetes/<p>Let me get this out first. There is small chance that what you need is Kubernetes, but it is highly unlikely.</p>
<blockquote>
<p>What you need is not Kubernetes. What you need is scaleable deployments.</p>
</blockquote>
<p>If you haven't already been able to tell, this is more of a rant blog.<br />
Just my opinions. Yup, I have them.</p>
<p>Let me explain how this went for me. I was dealing with a lot of scaleablility issues related to deploying stuff. There
was very little automated deployment setup for most of what I did as well. So when I initially heard about Kubernets I was excited as it
was about time I fix the automated deployment pipeline as well as scaleability issues.</p>
<p>Initially when I heard about Kubernetes, I was pretty much expecting to write a yaml file and have all my scaleability
problems vanish. But, the more I worked with Kubernetes, the more I realized that this is not true. Sure, Kubernetes is
one layer of abstraction on top just spinning up VMs and deploying your software on them. But it is not the final
solution to all of your problems.</p>
<h1 id="an-example-situation" tabindex="-1">An example situation <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#an-example-situation">#</a></h1>
<p>OK, in order to better explain this, let us consider a "hypothetical" situation. You have a frontend which is written in
the latest and greatest javascript framework (obviously). You also have a backend which is written is Python, nah Rust. Whatever is that you like.</p>
<p>You had a fun time developing them in your local machine and now you wanna deploy them.</p>
<h2 id="naive-approach" tabindex="-1">Naive approach <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#naive-approach">#</a></h2>
<p>Well, let us assume we go back in time(well Rust and that shiny js framework is not available in this time, but let us just ignore that time travel mess for now).
Here is how we would go about deploying our stuff. We buy a computer... Wait, maybe not that back in time. We spin up a
new VM on GCP or EC2. We install everything that you need to run the stuff. Node to build your static assests, or you
might even just only copy over the static files. You start your backend after installing python reqs if it is python.
After all this, you start your nginx reverse proxy and maybe even certbot for that sweet sweet https.</p>
<p>This works, and if we are not really concerned about scaling to a huge audience, this is more or less what you need.
But you had bigger plans for your app. You need to make sure it scales.</p>
<h2 id="kubernetes" tabindex="-1">Kubernetes <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#kubernetes">#</a></h2>
<p>Let us come back to current time period. Now we have Kubernetes, and we decide to deploy the app our new cluster.</p>
<p>You watch a bunch of tutorials and write that yaml file, or just copy paste it from somewhere and change some values.
You create yaml files for both frontend and backend.</p>
<p>Now you do <code>k apply -f myapp.yaml</code> and you are good. You see pods starting, stuff getting setup. You make a request, it work. You spam it with requests, it scales.</p>
<p>Well, at this point you think you have figured it all. But it is just starting.</p>
<h1 id="issues-you-will-run-into" tabindex="-1">Issues you will run into <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#issues-you-will-run-into">#</a></h1>
<h2 id="scaling-factor" tabindex="-1">Scaling factor <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#scaling-factor">#</a></h2>
<p>Well, when you wrote that yaml file before, you did not just specify that it should scale, but by what metric that it should scale as well.
One common metric that you see getting used is by CPU usage. This works for a lot of stuff. But(there is always a but),
this is not always the case. What if you have a service that pretty much just calls a few other services and aggregates
results and return them back(well, async.. shhh...). In this case, you would not see a CPU spike, so Kubernetes would not scale and you would not get any response back if your backend can only serve a few connection at a time.
This is only one situtaion, you will run into a lot of similar class of problems.</p>
<h2 id="adding-more-resources" tabindex="-1">Adding more resources <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#adding-more-resources">#</a></h2>
<p>Just because you have Kubernetes does not mean you have infinite resoureces.
I know you can spin up and attach new VMs to your cluster, but spinning up new VMs take some time and you cannot wait
on it if you have a really spikey traffic situation.</p>
<p>Also scaleing down to zero is something which is not directly available in Kubernetes.</p>
<h2 id="it-is-just-a-lot-of-yaml-copy-pasting" tabindex="-1">It is just a lot of yaml copy pasting <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#it-is-just-a-lot-of-yaml-copy-pasting">#</a></h2>
<p>Well, this is not in itself a big problem, but this should be an indicator that there is a problem. In a lot of cases, for
a backend deployment most of what you are doing with kubernetes is asking it to run a docker container and expose a
port. Attach a service to it and expose that to the internet.</p>
<p>So, if you keep stuff like ports consistent, you are pretty much only changing which docker image to use.
Nobody should be asked to write yaml for doing something like this.</p>
<h1 id="better-solutions" tabindex="-1">Better solutions <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#better-solutions">#</a></h1>
<p>This is not an exhaustive guide. Just some things I ended up using.</p>
<h2 id="functions-and-infinite-scaling" tabindex="-1">Functions and infinite scaling <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#functions-and-infinite-scaling">#</a></h2>
<p>Well, Kubernetes is pretty freaking awesome. But most of times, it is not enough abstraction for most people. What you
need is a way to say "Take this app and deploy it. PS: make it automatically scale". We are almost there, we can say.
"Deploy this docker image and auto scale it.", which I would argue is pretty close. For stuff like this what you
need is not Kubernetes, what you need is something like cloud-run or a similar service.
You might have have heard about stuff like knative, faas, riff which are pretty much this. They abstract away most of
that kube yaml from you and give you something better. There are other tools like Rancher which pretty much take all the yaml out of the equation.</p>
<h2 id="dedicated-tools" tabindex="-1">Dedicated tools <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#dedicated-tools">#</a></h2>
<p>Also, there are lot of people working on specialized tools to do specialized tasks. I would like to give you the
example of something like Netlify or Vercel for static assets like our frontend app.</p>
<p>In our sweet little app, once you are done with the build step, all you have remaining is to just serve static files.
You could technically put all of this static assets into and nginx container and ask stuff like cloud-run to scale that
nginx container image. But there are better options.</p>
<p>For example, if you were to you a service like Netlify, all you need to do is to specify what is the command to build your project, and which dir all your assets will be in after the build.
Once you have that done, they take you app, "scale" the crap out of it using their global CDNs. You are not beating that with anything on you Kube cluster.
Also in the case of Netlify, they recently introduced plugins which you can add to your build step to do tasks like
optimize images etc. And you even get live previews for PRs which is freaking epic.</p>
<p>This is only one example. In the case of Vercel, you can actually use Next.js and also have backend logic which they
scale as if each route was a function. Pretty neat, huh?</p>
<h1 id="appendix" tabindex="-1">Appendix <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#appendix">#</a></h1>
<p>There is more to this obviously, just wanted to let you know that Kubernetes is not the best option for all the problems out there.</p>
<p>Also there are a lot of things that help improve your k8s experience like helm.</p>
<p>It kinda feels weird saying this, but Kubernetes is a low level abstraction. But in case of most people it really is the truth.</p>
<p>That said, as I mentioned in the start there are reasons why you would need to use Kubernetes itself. But in most
cases, it is if you need more fine grained control over how, when and where things run.</p>
<h1 id="external-links" tabindex="-1">External links <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#external-links">#</a></h1>
<p>Here are a few things that that realated:</p>
<h3 id="tools%3A" tabindex="-1">Tools: <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#tools%3A">#</a></h3>
<ul>
<li>Helm: <a href="https://helm.sh/">https://helm.sh/</a></li>
<li>Knative: <a href="https://knative.dev/">https://knative.dev/</a></li>
<li>cloud-run: <a href="https://cloud.google.com/run/">https://cloud.google.com/run/</a></li>
<li>Rancher: <a href="https://rancher.com/">https://rancher.com/</a></li>
<li>riff: <a href="https://projectriff.io/">https://projectriff.io/</a></li>
<li>Netlify: <a href="https://www.netlify.com/">https://www.netlify.com/</a></li>
<li>Vercel: <a href="https://vercel.com/">https://vercel.com/</a></li>
</ul>
<h3 id="youtube-%26-podcast%3A" tabindex="-1">Youtube & Podcast: <a class="direct-link" href="https://blog.meain.io/2020/you-might-not-need-kubernetes/#youtube-%26-podcast%3A">#</a></h3>
<ul>
<li><a href="https://softwareengineeringdaily.com/2020/05/29/kubernetes-vs-serverless-with-matt-ward/">https://softwareengineeringdaily.com/2020/05/29/kubernetes-vs-serverless-with-matt-ward/</a></li>
<li><a href="https://realpython.com/podcasts/rpp/14/">https://realpython.com/podcasts/rpp/14/</a></li>
<li><a href="https://kubernetespodcast.com/episode/102-helm-graduation/">https://kubernetespodcast.com/episode/102-helm-graduation/</a></li>
<li><a href="https://www.youtube.com/watch?v=8age_72M_NE">https://www.youtube.com/watch?v=8age_72M_NE</a></li>
<li><a href="https://www.youtube.com/watch?v=E0GBU8Q-VFY">https://www.youtube.com/watch?v=E0GBU8Q-VFY</a></li>
<li><a href="https://www.youtube.com/watch?v=6sDTB4eV4F8">https://www.youtube.com/watch?v=6sDTB4eV4F8</a></li>
<li>Anything by Kelsey Hightower</li>
</ul>
Floating scratch terminal in tmux2020-09-15T00:00:00Zhttps://blog.meain.io/2020/tmux-flating-scratch-terminal/<p>Hi, Just another one off blog.
I have been using tmux for a while and one main thing I always wanted to have in tmux is a floating scratch terminal.
I got so used to this during my time with i3 and wanted to replicate it with just tmux.</p>
<p><img src="https://blog.meain.io/img/tmux-floating.png" alt="screenshot" /></p>
<blockquote>
<p>Check out what I am talking about on <a href="https://youtu.be/PdL__5AydVE">Youtube</a>.<br />
Code used here: <a href="https://github.com/meain/dotfiles/blob/64c9fcb00d0ce0ef3af96426cda84db5cd64f6e8/scripts/.local/bin/helpers/%2Cpopup-tmux">script</a> and <a href="https://github.com/meain/dotfiles/blob/master/tmux/.tmux.conf">tmux-config</a></p>
</blockquote>
<p>A while back, tmux actually got <a href="https://github.com/tmux/tmux/issues/1842">foating window</a> support and I am using it for a <a href="https://github.com/meain/dotfiles/blob/8523ac959e440e7d17e69507710ae85c200eea09/tmux/.tmux.conf#L216-#L231">lot of things</a>.
I had by this time had a floating terminal setup however and so did not really think about using this initially.</p>
<p>Today, I thought I would actually try getting this can be done in tmux. Just for the heck of it.</p>
<blockquote>
<p><strong>This is not yet in any released version just yet and you will have to build from master branch. It will be available in the 3.2 release.</strong></p>
</blockquote>
<h2 id="floating-windows-in-tmux" tabindex="-1">Floating windows in tmux <a class="direct-link" href="https://blog.meain.io/2020/tmux-flating-scratch-terminal/#floating-windows-in-tmux">#</a></h2>
<p>Basically you can get a floating window in tmux using the following command:</p>
<pre><code>tmux popup -R "ping meain.io"
</code></pre>
<p>This will start a ping to '<a href="http://meain.io/">meain.io</a>' in a floating window. Step one complete.</p>
<p>Now if you want to run something like a shell, you can use:</p>
<pre><code>tmux popup -KER zsh
</code></pre>
<blockquote>
<p>K & R is so that you can get input in to the process in the floating window<br />
E is so that after clean exit, we return back</p>
</blockquote>
<h2 id="persisting-session" tabindex="-1">Persisting session <a class="direct-link" href="https://blog.meain.io/2020/tmux-flating-scratch-terminal/#persisting-session">#</a></h2>
<p>Now that we have a terminal, we can technically use the tool that we have been using to persist stuff to persist the session.
When creating a popup, we can start a tmux session and attach to it on further invocations.
To quit out of the popup, we can just detach from it.</p>
<pre class="language-shell"><code class="language-shell">tmux popup <span class="token parameter variable">-KER</span> <span class="token string">"tmux attach -t popup || tmux new -s popup"</span></code></pre>
<p>The above script with attach to a session called popup, or create one if it does not exist.
We are half way there, but I don't wanna be pressing two different keys for showing and hiding the popup terminal.</p>
<p>For starters, let us create a script called <code>popuptmux</code> and put what we have in it.</p>
<p><strong>pouptmux</strong></p>
<pre class="language-shell"><code class="language-shell">tmux popup <span class="token parameter variable">-KER</span> <span class="token string">"tmux attach -t popup || tmux new -s popup"</span></code></pre>
<p>Now call this script with a tmux keybinding:</p>
<pre><code>bind-key j run-shell 'popuptmux'
</code></pre>
<p>Now when you press <kbd><prefix></kbd><kbd>j</kbd> it opens the session named <code>popup</code> in a floating window.
Now to make our script a bit more intelligent.</p>
<p><strong>pouptmux</strong></p>
<pre class="language-shell"><code class="language-shell"><span class="token keyword">if</span> <span class="token punctuation">[</span> <span class="token string">"<span class="token variable"><span class="token variable">$(</span>tmux display-message <span class="token parameter variable">-p</span> <span class="token parameter variable">-F</span> <span class="token string">"#{session_name}"</span><span class="token variable">)</span></span>"</span> <span class="token operator">=</span> <span class="token string">"popup"</span> <span class="token punctuation">]</span><span class="token punctuation">;</span><span class="token keyword">then</span><br /> tmux detach-client<br /><span class="token keyword">else</span><br /> tmux popup <span class="token parameter variable">-KER</span> <span class="token string">"tmux attach -t popup || tmux new -s popup"</span><br /><span class="token keyword">fi</span></code></pre>
<p>With this script, when you are in a session called <code>popup</code>(which you are when you have the floating window open) we detach,
otherwise we create a popup with a session named <code>popup</code> and attach to it.</p>
<p>Btw, if you want a bigger floating window, you can always just ask tmux.</p>
<p><strong>pouptmux</strong></p>
<pre class="language-shell"><code class="language-shell"><span class="token assign-left variable">width</span><span class="token operator">=</span><span class="token variable">${2<span class="token operator">:-</span>80<span class="token operator">%</span>}</span><br /><span class="token assign-left variable">height</span><span class="token operator">=</span><span class="token variable">${2<span class="token operator">:-</span>80<span class="token operator">%</span>}</span><br /><span class="token keyword">if</span> <span class="token punctuation">[</span> <span class="token string">"<span class="token variable"><span class="token variable">$(</span>tmux display-message <span class="token parameter variable">-p</span> <span class="token parameter variable">-F</span> <span class="token string">"#{session_name}"</span><span class="token variable">)</span></span>"</span> <span class="token operator">=</span> <span class="token string">"popup"</span> <span class="token punctuation">]</span><span class="token punctuation">;</span><span class="token keyword">then</span><br /> tmux detach-client<br /><span class="token keyword">else</span><br /> tmux popup <span class="token parameter variable">-d</span> <span class="token string">'#{pane_current_path}'</span> <span class="token parameter variable">-xC</span> <span class="token parameter variable">-yC</span> -w<span class="token variable">$width</span> -h<span class="token variable">$height</span> <span class="token parameter variable">-K</span> <span class="token parameter variable">-E</span> <span class="token parameter variable">-R</span> <span class="token string">"tmux attach -t popup || tmux new -s popup"</span><br /><span class="token keyword">fi</span></code></pre>
<blockquote>
<p>Checkout the discussion on <a href="https://www.reddit.com/r/tmux/comments/itonec/floating_scratch_terminal_in_tmux/">reddit</a>,
have a great workflow with floating terminals by <a href="https://www.reddit.com/user/KevinHwang91/">/u/KevinHwang91</a>
in <a href="https://www.reddit.com/r/tmux/comments/itonec/floating_scratch_terminal_in_tmux/g5jxke4">there</a>.</p>
</blockquote>
<p>All good, now we can just go on hitting <kbd><prefix></kbd><kbd>j</kbd> to open and close the floating window.
Although we do have this now, the performance ain't that good. So at the end of the day if you open and close it a lot, this might not be for you.</p>
Just a bunch of git stuff2020-11-06T00:00:00Zhttps://blog.meain.io/2020/bunch-of-git-stuff/<p>Hey,</p>
<p>Been a while since I have wrote a blog and thought I would write about something that I really like, git. I really like git, almost all of my important data is text and almost all of them are versioned using git. Here is a bunch of things in git that I think are not that widely known.</p>
<h1 id="worktree" tabindex="-1">Worktree <a class="direct-link" href="https://blog.meain.io/2020/bunch-of-git-stuff/#worktree">#</a></h1>
<p>Worktrees are useful if you work on a bunch of branches at the same time. It really comes in handy when you are in the middle of something in one branch have to switch to another branch to do a "urgent" bugfix.</p>
<p>I used to just clone the repo again, and work there and push, and delete that after it. It is kinda similar to this workflow, but better.</p>
<p>I also used to use another script which I call <a href="https://github.com/meain/dotfiles/blob/master/scripts/.bin/howwasit"><code>howwasit</code></a> which would technically checkout a branch in to a temporary directory so that I can see how something was at a commit. It is simliar to Github's browse repo at commit thing.
<img src="https://blog.meain.io/img/github-browse-at-commit.jpg" alt="Github screnshot" /></p>
<p>Enough preamble, lets get to the actual feature.
Here is how you can use it:</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">git</span> worktree <span class="token function">add</span> <span class="token operator"><</span>folder-name<span class="token operator">></span> <span class="token operator"><</span>branch-name<span class="token operator">></span> <span class="token comment"># existing branches</span><br /><span class="token function">git</span> worktree <span class="token function">add</span> <span class="token parameter variable">-b</span> <span class="token operator"><</span>new-branch<span class="token operator">></span> <span class="token operator"><</span>folder-name<span class="token operator">></span> <span class="token operator"><</span>tracking-branch<span class="token operator">></span> <span class="token comment"># new branches</span></code></pre>
<p>What the following command does is creates a "clone" of the repo at the branch that you specify to the folder that you specify.</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">git</span> worktree <span class="token function">add</span> <span class="token parameter variable">-b</span> new-branch new-stuff <span class="token number">0.1</span>.0</code></pre>
<p>Once you have a bunch of worktrees you can use the following command to see where all the worktrees are located.</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">git</span> worktree list<br />projects/lsd <span class="token number">3537575</span> <span class="token punctuation">[</span>master<span class="token punctuation">]</span><br />projects/lsd/new-stuff 0fd07cb <span class="token punctuation">[</span>new-branch<span class="token punctuation">]</span><br />projects/lsd/tree-stuff 0c53060 <span class="token punctuation">[</span>tree-d<span class="token punctuation">]</span></code></pre>
<p>From here, inorder to delete your worktrees, you can just delete the folder in which the worktree was created and call <code>git worktree prune</code> command. Calling <code>git worktree prune</code> will drop all the metadata that git had created for the deleted worktrees. Git will however keep the branch that you were using with the worktree however btw. You can drop that with <code>git branch -D <branch></code> as usual.</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">rm</span> new-stuff<br />$ <span class="token function">git</span> worktree prune</code></pre>
<h1 id="subtree" tabindex="-1">Subtree <a class="direct-link" href="https://blog.meain.io/2020/bunch-of-git-stuff/#subtree">#</a></h1>
<p>Next item on the list is <code>git subtree</code>. Think of this as <code>submodules</code> in git, but instead of just linking to an external repo, you pull in the code to your repo and add it via a commit. This can kinda be used as a low tech dependency managemnt tool like submodules, but you don't have to deal with any of the submodule issues. I always end up having issues with submodules in CI stuff.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">git</span> subtree <span class="token function">add</span> <span class="token parameter variable">--prefix</span><span class="token operator">=</span><span class="token operator"><</span>folder<span class="token operator">></span> <span class="token operator"><</span>repository<span class="token operator">></span> <span class="token operator"><</span>ref<span class="token operator">></span></code></pre>
<p>OK, let me show you an exmaple. Let us say that I am trying to create a very important JS library.</p>
<p>Here is the current state of the repo:</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">git</span> log <span class="token parameter variable">--oneline</span><br />e234947 <span class="token punctuation">(</span>HEAD -<span class="token operator">></span> master<span class="token punctuation">)</span> initial commit</code></pre>
<p>Now that we have that, I want to add my dependency <a href="https://github.com/jezen/is-thirteen">jezen/is-thirteen</a>. Here is how I would go about it.</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">git</span> subtree <span class="token function">add</span> <span class="token parameter variable">--prefix</span> deps/is-thirteen https://github.com/jezen/is-thirteen master <span class="token parameter variable">--squash</span><br /><span class="token function">git</span> fetch https://github.com/jezen/is-thirteen master<br />remote: Enumerating objects: <span class="token number">13</span>, done.<br />remote: Counting objects: <span class="token number">100</span>% <span class="token punctuation">(</span><span class="token number">13</span>/13<span class="token punctuation">)</span>, done.<br />remote: Compressing objects: <span class="token number">100</span>% <span class="token punctuation">(</span><span class="token number">13</span>/13<span class="token punctuation">)</span>, done.<br />remote: Total <span class="token number">1353</span> <span class="token punctuation">(</span>delta <span class="token number">6</span><span class="token punctuation">)</span>, reused <span class="token number">1</span> <span class="token punctuation">(</span>delta <span class="token number">0</span><span class="token punctuation">)</span>, pack-reused <span class="token number">1340</span><br />Receiving objects: <span class="token number">100</span>% <span class="token punctuation">(</span><span class="token number">1353</span>/1353<span class="token punctuation">)</span>, <span class="token number">4.79</span> MiB <span class="token operator">|</span> <span class="token number">69.00</span> KiB/s, done.<br />Resolving deltas: <span class="token number">100</span>% <span class="token punctuation">(</span><span class="token number">825</span>/825<span class="token punctuation">)</span>, done.<br />From https://github.com/jezen/is-thirteen<br /> * branch master -<span class="token operator">></span> FETCH_HEAD<br />Added <span class="token function">dir</span> <span class="token string">'deps/is-thirteen'</span></code></pre>
<p>The above command will add <code>is-thirteen</code> to <code>deps/is-thirteen</code>. My dir structure would look something like this.</p>
<pre><code>.
├── deps
│ └── is-thirteen
└── index.js
</code></pre>
<p>And this will be my commit log:</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">git</span> log <span class="token parameter variable">--oneline</span><br />29e241b <span class="token punctuation">(</span>HEAD -<span class="token operator">></span> master<span class="token punctuation">)</span> Merge commit <span class="token string">'05c2be39de99d98d8053c9531d63d1b449fec6e3'</span> as <span class="token string">'deps/is-thirteen'</span><br />05c2be3 Squashed <span class="token string">'deps/is-thirteen/'</span> content from commit ce004f9<br />e234947 initial commit</code></pre>
<p>If you check the commit log, you can see that the current state of the is-thirteen repo was squashed into that one commit and merged into our local repo. The squash happend becaused of the <code>--squash</code> flag in our command. If you were to avoid that, you will have the entire repo history of the <code>is-thirteen</code> repo in your repo as well.</p>
<p>Another usecase of git subtree is to split up something that you are working on into differnt repo. You can run something like below to do that.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">git</span> subtree <span class="token function">split</span> <span class="token parameter variable">--prefix</span><span class="token operator">=</span><span class="token operator"><</span>prefix<span class="token operator">></span> <span class="token operator"><</span>commit<span class="token operator">></span></code></pre>
<p>Here is an example. The following command will split your current repo's <code>ci</code> directory into its own branch with only those contents.</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">git</span> subtree <span class="token function">split</span> <span class="token parameter variable">--prefix</span><span class="token operator">=</span>ci <span class="token parameter variable">--branch</span> ci-stuff</code></pre>
<p>Now if you check the logs of the <code>ci-stuff</code> branch, you will only see commits that changed the <code>ci</code> directory. Technically this is new "repo". You can push this alone to a separate remote and continue from there. Comes in handly if you wanna split up some module for so that it can be used in multiple repos.</p>
<h1 id="log--l" tabindex="-1"><code>log -L</code> <a class="direct-link" href="https://blog.meain.io/2020/bunch-of-git-stuff/#log--l">#</a></h1>
<p>OK, now here is something amazing that git can do. I don't know if you have noticed it before but you might have seen that git when it shows diffs is actually able to show a context line at top which is usually a function or a class or something important like that. I initially thought it was just indent based, but turns out there is parsers for common things in common languages built into git.</p>
<p>In case you don't remember, let me remind you of that first.</p>
<pre class="language-diff"><code class="language-diff">@@ -41,22 +43,20 @@ class Helper:<br /><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> return data<br /></span><span class="token prefix unchanged"> </span><span class="token line"><br /></span></span><span class="token deleted-sign deleted"><span class="token prefix deleted">-</span><span class="token line"> def read_file(self, source, dataset):<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> def read_file(self, source, dataset, no_split=False):<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> if source.lower() == "fake_thing":<br /></span><span class="token prefix unchanged"> </span><span class="token line"> source = "unfake_thing_nah_just_kidding"</span></span></code></pre>
<p>If you check the above diff, you can see that it acutally gave me the class name as the context line.</p>
<p>This in itself is pretty neat, but turns out you can actually use this info to check the logs for just what has changed in that one function/class in all the commits. You can call <code>git log</code> with the <code>-L</code> flag and pass a specific function/class in a specific file and git will show what changes that specific thing has had.</p>
<p>For our examle here, we have to call it using something like below:</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">git</span> log <span class="token parameter variable">-L</span> :Helper:project/subdir/helper.py</code></pre>
<p>This will show all the changes that has happened to that python class and just that class alone.</p>
<h1 id="notes" tabindex="-1">Notes <a class="direct-link" href="https://blog.meain.io/2020/bunch-of-git-stuff/#notes">#</a></h1>
<p>OK, next one. <code>git notes</code>. No, this is not a note taking app with a git backend. We have enough of them already.</p>
<p>Well, I take a lot of notes. I really like taking notes. So, the idea with a note in git is that you can use it to add additional stuff to a commit message withouht adding anything to the actual commit message.</p>
<p>OK, I am pretty sure that confused you. Let me explain.
You can "attach" a note to any commit in your git history. Changing the note does not change the commit id(which means you can go on changing the note without having to rewording your commit). This is by default only stored locally but can be synced upstream if you prefer.</p>
<p>OK, enough gibberish. Let me show you how this actually works.
Let us work on our awesome js lib.</p>
<p>I will add a new note using the following command:</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">git</span> notes <span class="token function">add</span></code></pre>
<p>Once I have added, I can see all of my notes by doing <code>git notes</code>.</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">git</span> log <span class="token parameter variable">--oneline</span><br />29e241b <span class="token punctuation">(</span>HEAD -<span class="token operator">></span> master<span class="token punctuation">)</span> Merge commit <span class="token string">'05c2be39de99d98d8053c9531d63d1b449fec6e3'</span> as <span class="token string">'deps/is-thirteen'</span><br />05c2be3 Squashed <span class="token string">'deps/is-thirteen/'</span> content from commit ce004f9<br />e234947 initial commit<br /><br />$ <span class="token function">git</span> notes<br />34d30ee23154a1b060a523720c86a6e06a8c04bb 29e241b66ba494f4a31d7e9485f2afee4823f3bf</code></pre>
<p>This shows us that we added in a note with id <code>34d30ee23154a1b060a523720c86a6e06a8c04bb</code> to the commit <code>29e241b66ba494f4a31d7e9485f2afee4823f3bf</code> which is the merge commit. We can see our note using <code>git notes show <id></code></p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">git</span> notes show 29e241b66ba494f4a31d7e9485f2afee4823f3bf<br />So yeah, this is a note.<br />Just wanted to say we added <span class="token keyword">in</span> is-thirteen as a dependency.</code></pre>
<p>Git also shows you the note when you just look at the commit message.</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">git</span> show 29e241b<br />commit 29e241b66ba494f4a31d7e9485f2afee4823f3bf <span class="token punctuation">(</span>HEAD -<span class="token operator">></span> master<span class="token punctuation">)</span><br />Merge: e234947 05c2be3<br />Author: Abin Simon <span class="token operator"><</span>abinsimon10@gmail.com<span class="token operator">></span><br />Date: Fri Nov <span class="token number">6</span> <span class="token number">22</span>:07:55 <span class="token number">2020</span> +0530<br /><br /> Merge commit <span class="token string">'05c2be39de99d98d8053c9531d63d1b449fec6e3'</span> as <span class="token string">'deps/is-thirteen'</span><br /><br />Notes:<br /> So yeah, this is a note.<br /> Just wanted to say we added <span class="token keyword">in</span> is-thirteen as a dependency.</code></pre>
<p>Neat, right?</p>
<h1 id="commit---interactive" tabindex="-1"><code>commit --interactive</code> <a class="direct-link" href="https://blog.meain.io/2020/bunch-of-git-stuff/#commit---interactive">#</a></h1>
<p>Here is something that will let you do a bunch of stuff "interactively".
Well, I would not say that this is all that useful, at least was not to me personally. I would rather write a bunch of tiny scripts which lets me do this much more efficiently but this is a thing. Here is tiny preview into what this can do.</p>
<pre class="language-shell"><code class="language-shell">$ <span class="token function">git</span> commit <span class="token parameter variable">--interactive</span><br /> staged unstaged path<br /> <span class="token number">1</span>: unchanged +2/-0 index.js<br /><br />*** Commands ***<br /> <span class="token number">1</span>: status <span class="token number">2</span>: update <span class="token number">3</span>: revert <span class="token number">4</span>: <span class="token function">add</span> untracked<br /> <span class="token number">5</span>: patch <span class="token number">6</span>: <span class="token function">diff</span> <span class="token number">7</span>: quit <span class="token number">8</span>: <span class="token builtin class-name">help</span><br />What now<span class="token operator">></span> <span class="token number">5</span><br /> staged unstaged path<br /> <span class="token number">1</span>: unchanged +2/-0 index.js<br />Patch update<span class="token operator">>></span> i<br /> staged unstaged path<br />* <span class="token number">1</span>: unchanged +2/-0 index.js<br />Patch update<span class="token operator">>></span> <br /><span class="token function">diff</span> <span class="token parameter variable">--git</span> a/index.js b/index.js<br />index e69de29<span class="token punctuation">..</span>26786f1 <span class="token number">100644</span><br />--- a/index.js<br />+++ b/index.js<br />@@ -0,0 +1,2 @@<br />+const isThirteen <span class="token operator">=</span> require<span class="token punctuation">(</span><span class="token string">'is-thirteen'</span><span class="token punctuation">)</span><br />+is<span class="token punctuation">(</span><span class="token string">'പതിമൂന്ന്'</span><span class="token punctuation">)</span>.thirteen<span class="token punctuation">(</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token number">1</span>/1<span class="token punctuation">)</span> Stage this hunk <span class="token punctuation">[</span>y,n,q,a,d,e,?<span class="token punctuation">]</span>? y<br /><br />*** Commands ***<br /> <span class="token number">1</span>: status <span class="token number">2</span>: update <span class="token number">3</span>: revert <span class="token number">4</span>: <span class="token function">add</span> untracked<br /> <span class="token number">5</span>: patch <span class="token number">6</span>: <span class="token function">diff</span> <span class="token number">7</span>: quit <span class="token number">8</span>: <span class="token builtin class-name">help</span><br />What now<span class="token operator">></span> <span class="token number">1</span><br /> staged unstaged path<br /> <span class="token number">1</span>: +2/-0 nothing index.js<br /><br />*** Commands ***<br /> <span class="token number">1</span>: status <span class="token number">2</span>: update <span class="token number">3</span>: revert <span class="token number">4</span>: <span class="token function">add</span> untracked<br /> <span class="token number">5</span>: patch <span class="token number">6</span>: <span class="token function">diff</span> <span class="token number">7</span>: quit <span class="token number">8</span>: <span class="token builtin class-name">help</span><br />What now<span class="token operator">></span> </code></pre>
<p>Well, I am not sure what else to explain about other that it lets you "interactively" do stuff. Try it out next time when you have to do a commit. At any point just type <code>?</code> to get help as to what to do at that point.</p>
<h1 id="bonus" tabindex="-1">Bonus <a class="direct-link" href="https://blog.meain.io/2020/bunch-of-git-stuff/#bonus">#</a></h1>
<p>Woo, bonus. Well, these are some other things which are neat but are not technically within git. That is why I thought I would add this in a bonus section.</p>
<h2 id="spinoff" tabindex="-1">spinoff <a class="direct-link" href="https://blog.meain.io/2020/bunch-of-git-stuff/#spinoff">#</a></h2>
<p>I think this is a somewhat known thing to a lot of people, but I thougt I would include it anyways. Ever created a bunch of commits on master and was like, woops, should have created a new branch and submitted a PR for this instead of pushing directly to master. Well, this lets you do exactly that.</p>
<p>What this does is, creates a new branch at the current poinnt and rewinds the current branch back to what is in origin. I think this actually came from <a href="https://magit.vc/">magit</a> which btw is a sick frontend to git. Apparently this is popular thing now(it is pretty useful). <a href="https://github.com/nvie/git-toolbelt/blob/master/git-spinoff">Here</a> is a bash script which does the same thing.</p>
<h2 id="absorb" tabindex="-1">absorb <a class="direct-link" href="https://blog.meain.io/2020/bunch-of-git-stuff/#absorb">#</a></h2>
<p>Another entry in random useful stuff. <a href="https://github.com/tummychow/git-absorb">tummychow/git-absorb</a> is the repo where you have the "thing" that you need to do this.</p>
<p>Let me explain a sample situation in which you would be using this. You are going on a committing spree. It is all fun, but after a while you have to make a tiny edit which was ideally supposed to go into a previous commit and not a separate new commit.</p>
<p>What you could do with base git is that you will have to create a fixup commit with the proper commit to "absorb" these changes into. With this tool, all you have to do is to stage the commits that you wanna "absorb" and then call this tool. It will pick the correct commit to fixup this against.</p>
<p>From what I can tell, this looks for which commit changed the same parts of the file and creates a fixup commit against that. Kinda handy at times.</p>
<p>K. Thx. Bye.</p>
Highlight yanked region in Emacs2020-11-20T00:00:00Zhttps://blog.meain.io/2020/emacs-highlight-yanked/<p>OK, so I recently switched over to Emacs just becuse I woke up one day and got the itch to write some lisp.
It has been pretty fun so far.</p>
<p>Now with that out of the way, let me tell you about what I was actually planning to write about.
The idea is to show you a small example of how I am able to hack together stuff in Emacs.</p>
<blockquote>
<p>I know someone will ask for it, so here are my <a href="https://github.com/meain/dotfiles">dotfiles</a>. Steal away.</p>
</blockquote>
<p>Before switching to Emacs I have been a very heavy (neo)vim user for a long time.
And at some point I had the following snippet added from some reddit thread to my vimrc.</p>
<pre class="language-vim"><code class="language-vim"><span class="token builtin">autocmd</span> TextYankPost <span class="token operator">*</span> <span class="token keyword">silent</span><span class="token operator">!</span> lua <span class="token keyword">vim</span><span class="token operator">.</span><span class="token builtin">highlight</span><span class="token operator">.</span><span class="token function">on_yank</span><span class="token punctuation">(</span><span class="token punctuation">{</span>higroup<span class="token operator">=</span><span class="token string">"IncSearch"</span><span class="token punctuation">,</span> <span class="token builtin">timeout</span><span class="token operator">=</span><span class="token number">200</span><span class="token punctuation">}</span><span class="token punctuation">)</span></code></pre>
<p>What this script did was to highlight the region that I just yanked. This was pretty useful in my transition from doing <code>viwy</code> to <code>yiw</code>.
Just a way to verify that I yanked the correct thing. Let me show you what I am talking about.</p>
<p><img src="https://blog.meain.io/img/highlight.gif" alt="GIF of highlight on yank" /></p>
<p>In the above GIF, I yank the word under the cursor with <code>yiw</code> and so it highlights that. I have tweaked <code>word</code> to mean <code>symbol</code> in evil mode.
That is why it is selecting the full symbol and not just <code>-</code> which would have been the case in vim.</p>
<p>So, ever since I switch to Emacs, I have wanted to recreate this.
Recenly I came accross an <a href="https://karthinks.com/software/batteries-included-with-emacs/">article</a> by Karthinks which introduced me to <a href="https://www.emacswiki.org/emacs/PulseRegion"><code>pulse.el</code></a>.
This was pretty much what I was looking for. Now I just need to make this do the highlight on yank thing.</p>
<p>OK, so here is how I approached it.
In the blog, he mentions about <code>pulse-momentary-highlight-one-line</code> and I knew there had to be a pulse region command.</p>
<p>I started typing <code>pulse-momentary-</code> and by then Emacs had shown me <code>pulse-momentary-highlight-region</code> as as autocomplete option.
Cool, that looks like what I want. I opened up the documentation for that I saw that it takes a <code>begin</code> and <code>end</code> and an optional <code>face</code> in case you wanna change that.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token car">pulse-momentary-highlight-region</span> START END &optional FACE<span class="token punctuation">)</span></code></pre>
<p>Now with that figured out, I had to find how evil figures out the start and end location when I do a yank.
I started off with checking out what command gets called when do something with yank.</p>
<p>Hit <kbd>C-h</kbd><kbd>k</kbd><kbd>y</kbd> and there you go.
This told me that the function that I need to use is <code>evil-yank</code> and the params for that are as follows.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token car">evil-yank</span> BEG END &optional TYPE REGISTER YANK-HANDLER<span class="token punctuation">)</span></code></pre>
<p>Cool, looks like I can just add an advice to it. For those who don't know what an <code>advice</code> is, it pretty much just lets you wrap aroud a function with another.
Kinda like a python wrapper function.</p>
<p>Let me show you what the idea behind an advice is.
First you create a function which you use to add as an advice.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">my-advice-fun</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">orig-fn</span> <span class="token argument variable">arg1</span> <span class="token argument variable">arg2</span></span><span class="token punctuation">)</span></span><br /> <span class="token comment">;; Do things you wanna do before calling the function</span><br /> <span class="token punctuation">(</span><span class="token car">apply</span> orig-fn arg1 arg2<span class="token punctuation">)</span> <span class="token comment">;; Call the original function</span><br /> <span class="token comment">;; Do things you wanna do after calling the function</span><br /> <span class="token punctuation">)</span></code></pre>
<p>Now you ask the function to be added as an advice to some other function using <code>advice-add</code>.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token car">advice-add</span> the-function <span class="token lisp-property property">:around</span> <span class="token quoted-symbol variable symbol">#'my-advice-fun</span><span class="token punctuation">)</span></code></pre>
<p>This will add <code>my-advice-fun</code> as an advice to <code>the-function</code>.
OK, with that out of the way, let me show you what we need to do to get a highlight on yank.</p>
<p>First we create the function to be added as an advice.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/evil-yank-advice</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">orig-fn</span> <span class="token argument variable">beg</span> <span class="token argument variable">end</span> <span class="token rest-vars"><span class="token lisp-marker">&rest</span> <span class="token argument variable">args</span></span></span><span class="token punctuation">)</span></span><br /> <span class="token punctuation">(</span><span class="token car">pulse-momentary-highlight-region</span> beg end<span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">apply</span> orig-fn beg end args<span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p>Here we create an function called <code>meain/evil-yank-advice</code> which first calls <code>pulse-momentary-highlight-region</code> with the <code>begin</code> and <code>end</code> params
and after that calls the <code>orig-fn</code> with the same <code>begin</code> and <code>end</code>. One other thing we have here is <code>&rest args</code> which pretty much means you collect everything that is not <code>orig-fn</code>, <code>beg</code> or <code>end</code> to <code>rest</code>.
This way we can easily pass all those optional arguments that <code>evil-yank</code> will need to it without being aware of them explicitly.</p>
<p>Now to add this advice:</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token car">advice-add</span> <span class="token quoted-symbol variable symbol">'evil-yank</span> <span class="token lisp-property property">:around</span> <span class="token quoted-symbol variable symbol">'meain/evil-yank-advice</span><span class="token punctuation">)</span></code></pre>
<p>Voilà, you have highlight on yank.</p>
My workflow hacks2020-12-31T00:00:00Zhttps://blog.meain.io/2020/my-workflow-hacks/<p>Well, the year is about to end. It was a fun year, no shit. I wanted to write something like this for quite some time. I was like, "too lazy today, tomorrow" and the year is almost over. I did not want to keep this pending for next year and so started working on it.</p>
<p>Well, I just wanted to mention a bunch of thing that I have in my workflow to make my life a bit less miserable. I don't think anything specifically will be all that useful, but wanted to give an idea on the kind of things that you can do.</p>
<p><strong>Before we get into it, almost all of code for the things that gets mentioned in here will be available at <a href="https://github.com/meain/dotfiles">meain/dotfiles</a>.</strong></p>
<p>With the "preface" done, lets get into the content.</p>
<p>I am gonna break this into a bunch of categories, but they are just vague markers. Almost all of them are interlinked in some way or the other.</p>
<blockquote>
<p>I use macOS, so some of these things will be more or less macOS specific.</p>
</blockquote>
<h1 id="karabiner-elements" tabindex="-1">Karabiner-Elements <a class="direct-link" href="https://blog.meain.io/2020/my-workflow-hacks/#karabiner-elements">#</a></h1>
<blockquote>
<p>Link to webpage: <a href="https://karabiner-elements.pqrs.org/">https://karabiner-elements.pqrs.org/</a></p>
</blockquote>
<p>First item in the list is <a href="https://karabiner-elements.pqrs.org/">Karabiner-Elements</a>. It lets you do key remaps, some seriously crazy useful remaps might I add. It is kinda like <a href="https://github.com/qmk/qmk_firmware">qmk</a> but works on a OS level.</p>
<p>For starters, it lets you do simple key remaps, for example you can remap <kbd>caps lock</kbd> to <kbd>esc</kbd>. Very useful in Vim. But wait, don't do that yet. There is better things that you can do.
You can actually map the <kbd>caps lock</kbd> key to <kbd>esc</kbd> and <kbd>ctrl</kbd>. Well, the idea here is that if you just hit <kbd>caps lock</kbd>, it sends escape. But if you hold <kbd>ctrl</kbd> and hit a key like <kbd>x</kbd>, it will send out <kbd>ctrl</kbd>+<kbd>x</kbd>.</p>
<p>Well, this is where I started. After that I mapped all modifier keys to something else when hit alone. For example <kbd>ctrl</kbd> key send up arrow when hit alone. <em>I have to do this since I don't really have arrow keys on my keyboard (I use a 64 key keyboard).</em> But I don't use it that often, only if I only have one hand available like when watching Netflix while eating. I have <kbd>cmd</kbd>+<kbd>h</kbd>j<kbd>k</kbd><kbd>l</kbd> to be my arrow keys for the most part.</p>
<p>Btw, you can have any key do this, for example I have <kbd>'</kbd> mapped to send <kbd>ctrl</kbd> when pressed with another key. I don't have right <kbd>ctrl</kbd> as I have use it for something else plus even if I had it, it would be weird to reach that key.</p>
<p>There are even more stuff that you can do with <a href="https://karabiner-elements.pqrs.org/">Karabiner-Elements</a>. You can make <kbd>cmd</kbd>+<kbd>q</kbd> work only if you press and hold it and not just press it. It helps you from accidentally quitting something you were working in.</p>
<p>You can even just use two modifier keys. For example I have it so that if I hold left <kbd>shift</kbd> and hit right <kbd>shift</kbd> it switches me to the right monitor and if I hold right <kbd>shift</kbd> and hit left <kbd>shift</kbd> it switches me to the left monitor.</p>
<p>Btw, it can also control your mouse. You can control your mouse with just the keyboard if you want to. It gets crazy after a while. You can check all kinds of configuration examples at <a href="https://ke-complex-modifications.pqrs.org/">https://ke-complex-modifications.pqrs.org/</a> .</p>
<h1 id="hammerspoon" tabindex="-1">Hammerspoon <a class="direct-link" href="https://blog.meain.io/2020/my-workflow-hacks/#hammerspoon">#</a></h1>
<blockquote>
<p>Link to webpage: <a href="https://www.hammerspoon.org/">https://www.hammerspoon.org/</a></p>
</blockquote>
<p>If you thought <a href="https://karabiner-elements.pqrs.org/">Karabiner-Elements</a> was awesome, wait for <a href="https://www.hammerspoon.org/">Hammerspoon</a>. It is a way to control a lot of macOS "stuff" via lua. It lets you move windows, rearrange spaces, click on things, type things out, check bluetooth, wifi, kbd, mouse status and notify.... and a lot of other things.</p>
<p>Again, lets start with a bunch of simple things. You can map a keybinding to open/focus an app. From my personal collection I have <kbd>cmd</kbd>+<kbd>shift</kbd>+<kbd>k</kbd> to focus firefox.</p>
<p><em>Btw, just in case for anyone who is wondering(as if anyone reads my blog) where the <code>k</code> for firefox comes from it does not stand for anything. I just have <code>j</code>, <code>k</code>, <code>l</code> mapped to my text editor, browser and terminal respectively.</em></p>
<p>What I have for <kbd>cmd</kbd>+<kbd>shift</kbd>+<kbd>k</kbd> currently is a bit more involved. When I press, <kbd>cmd</kbd>+<kbd>shift</kbd>+<kbd>k</kbd> it focuses firefox if it is already open, if not launch it plus moves my cursor to the monitor in which it currently is on. Along with that, it retains which app I was on previously so that if I hit <kbd>cmd</kbd>+<kbd>shift</kbd>+<kbd>k</kbd> again when I am in firefox it switches me back to the app that I was on previously. It helps with quickly switching to the browser to check the docs and back with just hitting <kbd>cmd</kbd>+<kbd>shift</kbd>+<kbd>k</kbd> twice. Fun stuff right, I know.</p>
<p>Sometimes there is an action that you always do in a specific app, like open a new tab in a browser. I found my self hitting <kbd>cmd</kbd>+<kbd>t</kbd> when I am in my editor to open a new tab to search for some stupid thing that I was supposed to know. So I made <kbd>cmd</kbd>+<kbd>t</kbd> do exactly that. Now, it does not matter which app I am in when I hit <kbd>cmd</kbd>+<kbd>t</kbd>, it just pops me into the browser if I am not already in there and opens a new tab ready for me to type in a stupid query. I have a bunch of things like this. Another really good use for this is for Slack. Every time I get a new message, I have to switch to Slack, look up the the chat or even worse, use my mouse to click on the notification. Now all I have to do is hit <kbd>cmd</kbd>+<kbd>e</kbd> and from wherever I am and I end up in slack with a search window to open the "recent"(the one with the new message) chats.</p>
<p>Here is a small section dedicated to how I am dealing with Zoom calls aka "quick call that lasts 2 hours". First of all, about muting myself on a call. This is very important to me, if I am not talking, I am on mute. I work from home and well my home is not a workplace and so there is a lot of chance for a lot of noise. Plus I don't want anyone hearing me say, "woosh, when does this call get over". As you might have guessed, I use hammerspoon for this. <a href="https://www.hammerspoon.org/">Hammerspoon</a> has this concept of <a href="https://github.com/Hammerspoon/hammerspoon/blob/master/SPOONS.md">Spoons</a> which are essentially plugins and I got most of the code from the <a href="http://www.hammerspoon.org/Spoons/MicMute.html">MicMute</a> spoon. After some edits from my end, with a keybinding it lets me mute my mic from anywhere plus it shows me my current mic status in my status bar area along with a tiny alert every time I switch between mute and unmute. You can find my version of the code <a href="https://github.com/meain/dotfiles/blob/master/hammerspoon/.config/hammerspoon/Spoons/MicMute.spoon/init.lua">here</a>. Well, I have a bunch of other things for zoom like opening links directly in the app than going through browser, joining the meeting that is happening now with just one keybind etc but one other things that I do with purely hammerspoon is start new meetings. For me, with <kbd>alt</kbd>+<kbd>ctrl</kbd>+<kbd>shift</kbd>+<kbd>z</kbd> (this might look like it is hard to press, but I have a remap <kbd>alt</kbd>+<kbd>shift</kbd>) it opens zoom, starts a new meeting with my personal zoom id and drop the meeting link to the chat that I am in. Think of how many steps of pain and agony that reduces(at least 6-7). Well, gonna stop with Zoom stuff here.</p>
<p>One more thing under the hammerspoon section. I have a script/module called <a href="https://github.com/meain/dotfiles/blob/006b958b5ea2431bcb6e736b6ed5ebf5033d7103/hammerspoon/.config/hammerspoon/init.lua#L324">quick-edit</a>. The idea behind this is that, if you are in a text field and you hit a keybinding. It opens up the content of that text field in my editor, with markdown syntax highlight and let me edit with all the vim keybinding goodness(in <a href="https://www.gnu.org/software/emacs/">Emacs</a> using evil-mode. Huh, gotcha).</p>
<h1 id="fun-with-clipboard" tabindex="-1">Fun with clipboard <a class="direct-link" href="https://blog.meain.io/2020/my-workflow-hacks/#fun-with-clipboard">#</a></h1>
<p>Clipboard is something that you should not take for granted, it is a really powerful tool. It might look simple, but this is what will let you bridge a lot of gaps as everybody reads the clipboard and writes to it(it is a security nightmare, but it is useful). For example for a while I had a cron job which would drop the meetings ids from my calendar every time it is time to join a call. And so, if I do decide to join that call, all I have to do is press a single keybind. In this case <kbd>alt</kbd>+<kbd>backspace</kbd>. Well, the exact thing that I have mapped to <kbd>alt</kbd>+<kbd>backspace</kbd> is more of a link opener. If the link is associated with an app, it will open that app with the link or open that in a browser. If it is not a link, it searches for that thing in the browser with the search engine of your choice. A note on this is that you can always do transformation half way through. For example if what I have in the clipboard is a git ssh url, it converts that to https and opens that.</p>
<p>Btw, it is not like you are restricted to opening things. If I had a git ssh url in the clipboard and I called a command, it would to clone the repo to a temp folder and start a <a href="https://github.com/tmux/tmux">tmux</a> session there for me to work on that. The options are endless, you just have to figure out what you need to do.</p>
<p>Another thing you could do is just plain open up a list of things that you could do what is in the clipboard with options like bas64 decode or custom search engine list or something.</p>
<h1 id="note-taking-and-bookmarking" tabindex="-1">Note taking and bookmarking <a class="direct-link" href="https://blog.meain.io/2020/my-workflow-hacks/#note-taking-and-bookmarking">#</a></h1>
<p>Well, this is something that I have been constantly evolving. At the end of the day, it is mostly just a bunch of plain text files that I grep, sed and do other unspeakable things to. I, like a lot of other people have my own note taking script that I wrote in bash. For the most parts, it is a bunch of files with some sort of categorization. To look for a file, I just more or less use <a href="https://github.com/junegunn/fzf">fzf</a> as a completion system and open that file up in my editor. I use git + gitlab to backup my useless notes. I initially used to use git to keep my notes in sync with my phone as well, but now I just use <a href="https://syncthing.net/">Syncthing</a>. It is how syncing should be, just clean simple stuff, no BS.</p>
<p>Similarly for bookmarks, it is kind of a same deal. In this case it is a bunch of files with vauge categorization. Each line in a file is a bookmark and contains the link and a description that I put in. Again, I have <a href="https://github.com/junegunn/fzf">fzf</a> let me filter through this list. One addition that I think I should mention is that I have something that will go though all my links and curl the title from their webpage automatically. This lets me help "fzf" for it as well.</p>
<blockquote>
<p>"fzf for it" should be a thing like "grep for it".</p>
</blockquote>
<p>Another classification of "notes" that I have is scratchpads. These are essentially one off file(or so I thought) which I don't really backup. The idea is same for this as well. I have script called <code>vime</code> which will open up a text file with a random name(the name for the one used to write this blog's bullet points is <code>_6d5b</code>) in my text editor. I have, as you might have guessed it keybinds for this as well. No matter what useless things I am doing, I can jot down stuff about them.</p>
<h1 id="todo%2Ftask-management" tabindex="-1">Todo/task management <a class="direct-link" href="https://blog.meain.io/2020/my-workflow-hacks/#todo%2Ftask-management">#</a></h1>
<blockquote>
<p>Link to webpage: <a href="https://taskwarrior.org/">https://taskwarrior.org/</a></p>
</blockquote>
<p>Let me introduce to you another gem, <a href="https://taskwarrior.org/">Taskwarriror</a>. I am not a big GTD guy, but this kinda helps with "getting tings done" or at least reminding that I have things to be done. The best part about this is that this is a cli app, so how you choose to actually use it is up to you. I use taskwarrior a lot, but I never really use the cli. I have a bunch of hammerpoon "UIs" on top of it. I wanted to attach a screenshot here, but it right now has a lot of my personal and work tasks and I am really not in the mood to create a lot of dummy tasks to get a screenshot. I will link to the <a href="https://github.com/meain/dotfiles/blob/master/hammerspoon/.config/hammerspoon/taskwarrior.lua">code</a> though and you should be able to try it out. It essentially is just list of tasks plus an add button but a bit more intelligent on how tasks are added or marked as done.
I also make use of <code>hs.canvas</code> to have a desktop widget which list my most important tasks on my secondary monitor.</p>
<p>Btw, since this is a cli app. I have tasks that get automatically completed from a cron job or a script. A simple example is that it will automatically mark my task of writing a journal complete as soon I save and quit the journal file in my editor. Forgot to mention about my Journaling system. It is just the same as notes, but just that it will encrypted with gpg as soon as I am done editing. Not that it contains anything sensitive, just that I was paranoid that someone might read about me gushing over my cursh. And it is synced with Gitlab for backup btw.</p>
<p>And for the last part of syncing tasks/todo. Even though I am not really all that much of a going out all the time person, if a task management solution does not sync between all your devices, it is not all that useful for me. <a href="https://taskwarrior.org/">Taskwarriror</a> has a relatively good syncing setup. I use <a href="http://freecinc.com/">Freecinc</a> to sync <a href="https://taskwarrior.org/">Taskwarriror</a> stuff with my phone. I have used a bunch of mobile taskwarrior clients and to be frank, most of them look ugly as shit. Been experimenting with a few now, nothing that I would fully use. As of now, I use the <a href="https://taskwarrior.org/">Taskwarriror</a> cli in Termux. There is a lot to discuss about termux as well, but I am gonna leave that to the future me(I am guessing in another 30 mins). It works, plus I am able to share something like a link to termux and have it automatically added my reading list.</p>
<p>There is a lot more to this, I am gonna stop here though.</p>
<h1 id="email-workflow" tabindex="-1">Email workflow <a class="direct-link" href="https://blog.meain.io/2020/my-workflow-hacks/#email-workflow">#</a></h1>
<p>I love email. It is awesome. The best part about it is that nobody expects you to respond immediately. If you don't like email, it is not a problem with email, it a problem with how you consume email. My email workflow includes <a href="https://notmuchmail.org/">notmuch</a> and <a href="https://www.gnu.org/software/emacs/">Emacs</a>. <a href="https://notmuchmail.org/">Notmuch</a> is really useful. In its base form is a mail indexing thingy. It lets you filter, search, assign tags etc to your email. And since it is a cli application, you can script in a lot of things(notice the theme here). For starters, I have file which contains a lot of known addresses and these will get automatically tagged. Well, it is not like you have to add something to a file or update the script to tag stuff for every new email. You can automate that from your email viewer. For example in my case, if I have some special tags which if I ever mark an email with it, all future emails from them will go to that tag. I could have this filter on full email, domain or whatever I choose. An example use case of this would be for mailing lists. Not everything is interesting and you can just have it filter out a regex in the title with a tag and you never have to see any of those things again. Also, I have hammerspoon bindings for this to notify me on new email, or quickly view what/who I have email from, mark them as read and all that kinda stuff.</p>
<p>Another things which ideally deserves its own spot is RSS feeds, they are useful but for some reason not a lot of people seem to know/care about it anymore. They are a good way to subscribe to things/events. I don't want to get an email every time someone writes a blog article, I rather have this in my rss feeds for me to go through when I have time.</p>
<h1 id="kde-connect" tabindex="-1">KDE Connect <a class="direct-link" href="https://blog.meain.io/2020/my-workflow-hacks/#kde-connect">#</a></h1>
<p>This is something that I discovered recently. Well, I knew this existed, but never knew this was available for mac. But it is. It lets you connect your phone and your laptop a bit more. Like sync clipboard, send commands, send a receive files and all that kind of stuff. One awesome thing that this lets me do is if I am reading an article on my phone, I can share the link to the KDE Connect app on my phone and it will open that link in my computer. Just magical. Plus it does clipboard sync, and you know what I think about clipboards.</p>
<p>It also lets me run commands on my system with my phone. My primary usecases for this is to disconnect my headphone from my laptop when I am lying down my couch. My headphones can connect to two devices at a time, but I hate when it switches to the laptop audio when I get a completely useless message and I am watching something on my phone. Well, recude the annoying things. That is kinda the idea.</p>
<h1 id="termux" tabindex="-1">Termux <a class="direct-link" href="https://blog.meain.io/2020/my-workflow-hacks/#termux">#</a></h1>
<p>Termux is a useable terminal on your Android phone. Well, when I first heard about it, this sounded like a stupid idea. Don't get me wrong, I love terminals, but on a phone? Well, you see the point to the terminal is automating things. Plus maybe ssh into your laptop to check on something on run a script. I primarily use it for taskwarrior, bookmarks and ssh. The best part is that you get to use the tools that you have created good workflows with, like <a href="https://github.com/junegunn/fzf">fzf</a> on your phone. Ain't that sweet? Plus you can share links to this app and have it do something with it which again you can script.</p>
<p>Here is a "hypothetical" workflow. You have a link to a youtube video, you can send that to termux. Termux now ssh into some server, download that, encode it to a different video format or like speed it up and send it to you on telegram/matrix or just drop it direcly in the phone filesystem. Think of all the possibilities.</p>
<h1 id="byee" tabindex="-1">Byee <a class="direct-link" href="https://blog.meain.io/2020/my-workflow-hacks/#byee">#</a></h1>
<p>It is not one single things that make it useful. It is the fact that I can combine all of these tools together that help it make the final workflow powerful. I am gonna end it here, but I would welcome you to checkout my <a href="https://github.com/meain/dotfiles">dotfiles</a> repo. That will contain most of the stuff that I experiment on.</p>
<p>There are two other items that I did not specifically mention but are really important to my workflow. That is <a href="https://www.gnu.org/software/emacs/">Emacs</a> and <a href="https://github.com/tmux/tmux">Tmux</a>. These are really useful and is more or less involved in most of the things that I do. Just had to drop that in there. A lot of cli programs as well, but don't wanna list them all here now.</p>
<p>The idea was to introduce you to the kind of things that you can do and the tools that are available. Hopefully I have convinced at least someone to <s>use these things</s> think that I am not completely crazy for doing this.</p>
Just a simple markdown previewer2021-04-27T00:00:00Zhttps://blog.meain.io/2021/offline-markdown-preview/<p>I really like markdown. I write everything in markdown be it blogs, presentations, emails, todo lists, documentation, everything is in markdown. For the most part I can parse and render markdown in my head and don't need anything more than a simple <a href="https://www.gnu.org/software/emacs/">text editor</a> that will let me write stuff down. But, from time to time, I do like to render the markdown out and see how it would look.</p>
<p>I also use git a lot and as a result use Github a lot. So most of the markdown that I have seen rendered is in the Github format. It is simple and clean. I have previously used multiple things to render markdown when I wanted to view the rendered version but thought I need a good solution which I can use to render and view the markdown file locally. This is my journey to find/build that tool.</p>
<p>For those who need something that just works, checkout <a href="https://github.com/joeyespo/grip">grip</a>. It is great and the markdown that it generates looks exactly like what Github would generate. This is actually because it is Github that is genrating the markdown. <code>grip</code> <em>send a request to github and gets it rendered</em> and that is what gets displayed. This was a deal breaker for me for two reasons.</p>
<ul>
<li>It send everything that I write to Github</li>
<li>It has rate limiting (<a href="https://github.com/joeyespo/grip/issues/35">ref</a>)</li>
</ul>
<p>What I wanted was grip, but it does things offline. Before we dip into this, let me lay down what the things that I looking for was.</p>
<ul>
<li>Editor independent script</li>
<li>Does not need external service to render markdown</li>
<li>Automatically refresh page on save</li>
<li>Looks like Github's rendered version</li>
</ul>
<p>Let me show you how I went about it.</p>
<blockquote>
<p><strong>Code mentioned here are available on my Github. <a href="https://github.com/meain/dotfiles/blob/08bad8a5749cc6c61cad572a73100f853292c321/scripts/.bin/markdown-preview">script</a> and <a href="https://github.com/meain/dotfiles/blob/08bad8a5749cc6c61cad572a73100f853292c321/datafiles/.config/datafiles/pandoc-github-template.html">template</a></strong></p>
</blockquote>
<h1 id="generating-the-html-for-the-markdown" tabindex="-1">Generating the html for the markdown <a class="direct-link" href="https://blog.meain.io/2021/offline-markdown-preview/#generating-the-html-for-the-markdown">#</a></h1>
<p>I am pretty sure I knew how to do this. I have been using <a href="https://pandoc.org/">pandoc</a> for quite a while now. I love it and it should work great here as well. What I could do is just let pandoc take the raw markdown and render it to html. A simple conversion would go something along the lines of:</p>
<pre class="language-bash"><code class="language-bash">pandoc <span class="token parameter variable">-f</span> gfm <span class="token parameter variable">-t</span> html5 <span class="token parameter variable">--output</span> document.html document.md</code></pre>
<p>What this command is doing is taking <code>document.md</code> which is in <code>gfm</code>(Github flavoured markdown) and converting that to <code>html5</code> and storing in document.html. This step does get us the html that we need. Next step would be to style the thing to look like Github's rendering.</p>
<blockquote>
<p>Checkout pandoc even if you don't plan to use it for this. It is a great tool which you can use to convert from a lot of formats to lot of other formats. You can even convert markdown to Word/PDF documents to be sent to your product teams.</p>
</blockquote>
<p><img src="https://blog.meain.io/img/just-html.png" alt="Screenshot with just html conversion" /></p>
<p>This is how it would look like after this step. It gives us semantic html, but does not look all that pretty.</p>
<h1 id="styling-markdown" tabindex="-1">Styling markdown <a class="direct-link" href="https://blog.meain.io/2021/offline-markdown-preview/#styling-markdown">#</a></h1>
<p>This part was relatively easy. I just had to find some css that I can apply to the html doc to make it look like Github's. There was quite a few of these out there. What I ended up using was:</p>
<pre><code>https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/3.0.1/github-markdown.min.css
</code></pre>
<p>There was some extra templating that I had to add in as well to get it to look like it and center it. Pandoc actually lets us pass in the template and I made use of that here. This is the template that I used.</p>
<pre class="language-html"><code class="language-html"><span class="token doctype"><span class="token punctuation"><!</span><span class="token doctype-tag">DOCTYPE</span> <span class="token name">html</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>html</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>title</span><span class="token punctuation">></span></span>Markdown preview<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>title</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>link</span><br /> <span class="token attr-name">rel</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>stylesheet<span class="token punctuation">"</span></span><br /> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/3.0.1/github-markdown.min.css<span class="token punctuation">"</span></span><br /> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>body</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>article</span><br /> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>markdown-body<span class="token punctuation">"</span></span><br /> <span class="token special-attr"><span class="token attr-name">style</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><br /> box-sizing: border-box;<br /> min-width: 200px;<br /> max-width: 980px;<br /> margin: 0 auto;<br /> padding: 45px;<br /> <span class="token punctuation">"</span></span></span><br /> <span class="token punctuation">></span></span><br />$body$<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>article</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>body</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>html</span><span class="token punctuation">></span></span></code></pre>
<p>The <code>$body$</code> piece is where pandoc would dump the converted html. This get us a bit more further. To use a template in with pandoc we can call the binary like so.</p>
<pre class="language-shell"><code class="language-shell">pandoc <span class="token parameter variable">--template</span> template.html <span class="token parameter variable">-f</span> gfm <span class="token parameter variable">-t</span> html5 <span class="token parameter variable">--output</span> document.html document.md</code></pre>
<p><img src="https://blog.meain.io/img/md-styled.png" alt="Screenshot with styled html" /></p>
<p>Things are stating to look good, but where is my syntax highlighting.</p>
<h1 id="syntax-highlight" tabindex="-1">Syntax highlight <a class="direct-link" href="https://blog.meain.io/2021/offline-markdown-preview/#syntax-highlight">#</a></h1>
<p>Next piece is syntax highlight. This is something that still does not match 100% with what Github does, but it is pretty close. For this I make use of <a href="https://highlightjs.org/">highlight.js</a>. What this does is it pull down code parsers written in javascript and parses and highlights all the code blocks in the html. Pandoc acutally can do the code parsing as well and that is an option that I explored initially but decided to settle with this.</p>
<p>For this to work, we need to add two files to our template. A css file which will have the theme and a js file which wil be able to pull the other language specific js files needed for parsing. All we need to add here is just two more items to template.</p>
<pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>link</span> <span class="token attr-name">rel</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>stylesheet<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>https://unpkg.com/@highlightjs/cdn-assets@10.7.2/styles/github-gist.min.css<span class="token punctuation">"</span></span><span class="token punctuation">/></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>https://unpkg.com/@highlightjs/cdn-assets@10.7.2/highlight.min.js<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script"></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span></code></pre>
<p><em>If you want to make everything offline, you can just download these scripts and just have them load from local sources.</em></p>
<p>But this alone does not give us syntax highlighting. The issue is that <code>highlight.js</code> expects the <code><code></code> block to have the classes <code>hljs</code> and the language name but what pandoc does is put the language name as class name in <code><pre></code> block. We can just write some javascript to fix this.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="token keyword">let</span> ci <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">.</span><span class="token function">slice</span><span class="token punctuation">.</span><span class="token function">call</span><span class="token punctuation">(</span>document<span class="token punctuation">.</span><span class="token function">getElementsByTagName</span><span class="token punctuation">(</span><span class="token string">"pre"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />ci<span class="token punctuation">.</span><span class="token function">forEach</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">i</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /><span class="token keyword">let</span> children <span class="token operator">=</span> i<span class="token punctuation">.</span>children<span class="token punctuation">;</span><br /><span class="token keyword">if</span> <span class="token punctuation">(</span>children<span class="token punctuation">.</span>length <span class="token operator">===</span> <span class="token number">1</span> <span class="token operator">&&</span> children<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>tagName <span class="token operator">===</span> <span class="token string">"CODE"</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> cn <span class="token operator">=</span> i<span class="token punctuation">.</span>className<span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>cn<span class="token punctuation">.</span>length <span class="token operator">></span> <span class="token number">0</span><span class="token punctuation">)</span> i<span class="token punctuation">.</span>children<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>classList<span class="token punctuation">.</span><span class="token function">add</span><span class="token punctuation">(</span><span class="token string">"hljs"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>cn<span class="token punctuation">.</span>length <span class="token operator">></span> <span class="token number">0</span><span class="token punctuation">)</span> i<span class="token punctuation">.</span>children<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>classList<span class="token punctuation">.</span><span class="token function">add</span><span class="token punctuation">(</span>cn<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />hljs<span class="token punctuation">.</span><span class="token function">highlightAll</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>The above code pretty much goes through all <code><pre></code> tags and for all them that has a <code><code></code> block as its child applies the class <code>hljs</code> as well as whatever was on <code><pre></code> tag(which would be the language name). This would get it to the format that we need. At the end we also need to call <code>hljs.highlightAll()</code> to let hljs go and start highlighting the code blocks.</p>
<p><img src="https://blog.meain.io/img/md-syntax-highlighted.png" alt="Screenshot with syntax highlight" /></p>
<p>Finally, things are stating to look good.</p>
<h1 id="automatic-refresh" tabindex="-1">Automatic refresh <a class="direct-link" href="https://blog.meain.io/2021/offline-markdown-preview/#automatic-refresh">#</a></h1>
<p>This includes two pieces. One is listening for changes in the markdown file and regenerating the html file. Second one is to refresh the browser tab that is displaying the content.</p>
<p>First one is simple, all we need to use is a simple file watcher. I use <a href="https://github.com/meain/on-change">meain/on-change</a>, but there are a lot of great tools out there. My suggestions would be <a href="https://github.com/clibs/entr">entr</a> or <a href="https://github.com/watchexec/watchexec">watchexec</a>. These tools will let us run a command every time a file gets updated.</p>
<p>With <code>meain/on-change</code> what I can do would be:</p>
<pre class="language-shell"><code class="language-shell">on-change document.md <span class="token string">"pandoc --template template.html -f gfm -t html5 --output document.html document.md"</span></code></pre>
<p>What this means is every time <code>document.md</code> changes, rerun the command.</p>
<p>Now, for the second piece of this puzzle. Usually the patten here is to run a server and notify the browser through an open socket that there are updates and ask it to reload. But I did not want to bother running a server.</p>
<p>I kinda went for hacky approach for this. I have a script running in the page which will run every second. Every second, it pull the file and check if there is any change from what it got the previous time. If so, it updates the page.</p>
<p>About updating the page, initially if I found that there was a difference what I did was to just reload the browser. But this was causing some jitter. So a better way that I found was to just replace the entire <code><html></code> block in docuemnt with what I got as a response from fetch. This actually works pretty darn well. Below is the core of this code.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="token function">setInterval</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token function">fetch</span><span class="token punctuation">(</span>window<span class="token punctuation">.</span>location<span class="token punctuation">)</span><br /> <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">d</span><span class="token punctuation">)</span> <span class="token operator">=></span> d<span class="token punctuation">.</span><span class="token function">text</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">con</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token comment">// content will have the previous content</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>content <span class="token operator">!==</span> con<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> content <span class="token operator">=</span> con<span class="token punctuation">;</span><br /> document<span class="token punctuation">.</span><span class="token function">getElementsByTagName</span><span class="token punctuation">(</span><span class="token string">"html"</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> con<span class="token punctuation">;</span><br /> <span class="token function">fixCodeBlocks</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// runs the code that fixes <code> tags for hljs</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">1000</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>Now with this, all I have to do is to open up the file in the browser. I don't have to run any servers.
I can just add <code>open docuemnt.html</code> in the script and it would open up <code>file://document.html</code> in the browser and we are good to go.</p>
<p>Well, that is it for this one. Bye. Stay safe.</p>
Templating things in Airflow DAG2021-05-12T00:00:00Zhttps://blog.meain.io/2021/airflow-template-in-dag/<p>Just another, TIL type entry. It is about Airflow. I have been working with Airflow for quite a while. I don't really like it to be frank, but I am stuck with it for now.
With all that said, you can do everything that you want to do in Airflow. There is always some way to do everything. If you try really hard, you will be able to find even a way to capture Mewtwo. I am not saying this is something as important as that, but it took me a while to figure it out and I thought I should document it somewhere.</p>
<p>What I want to go over in this blog is ways in which we can get different variables into a dag which has KubernetesPodOperator.</p>
<h1 id="method-1---from-airflow-config" tabindex="-1">Method 1 - From airflow config <a class="direct-link" href="https://blog.meain.io/2021/airflow-template-in-dag/#method-1---from-airflow-config">#</a></h1>
<p>One easy place where we can pull a config from is the <code>airflow.cfg</code> file. This can be done with something like below:</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">from</span> airflow<span class="token punctuation">.</span>configuration <span class="token keyword">import</span> conf<br /><br /><span class="token keyword">with</span> DAG<span class="token punctuation">(</span><br /> dag_id<span class="token operator">=</span><span class="token string">"my-little-pony"</span><span class="token punctuation">,</span> schedule_interval<span class="token operator">=</span><span class="token boolean">None</span><span class="token punctuation">,</span> start_date<span class="token operator">=</span>YESTERDAY<br /><span class="token punctuation">)</span> <span class="token keyword">as</span> dag<span class="token punctuation">:</span><br /> head <span class="token operator">=</span> KubernetesPodOperator<span class="token punctuation">(</span><br /> image_pull_secrets<span class="token operator">=</span>conf<span class="token punctuation">.</span>get<span class="token punctuation">(</span><span class="token string">"kubernetes"</span><span class="token punctuation">,</span> <span class="token string">"image_pull_secrets"</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> image_pull_policy<span class="token operator">=</span><span class="token string">"Always"</span><span class="token punctuation">,</span><br /> name<span class="token operator">=</span><span class="token string">"build-head"</span><span class="token punctuation">,</span><br /> cmds<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"python"</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> arguments<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"build-head"</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><br /> <span class="token punctuation">)</span></code></pre>
<p>If you see the example above, you can see how the <code>image_pull_secrets</code> value is fetched form the config. This works great if we just want to reuse things from the airflow config. I mainly use this for <code>image_pull_secrets</code> and <code>namespace</code> as they were same as the airflow webserver in my case.</p>
<h1 id="method-2---airflow-variables" tabindex="-1">Method 2 - Airflow Variables <a class="direct-link" href="https://blog.meain.io/2021/airflow-template-in-dag/#method-2---airflow-variables">#</a></h1>
<p>Next item is <code>Variables</code>. These are values that can be set from the Airflow UI. It is also possible to set them via env variables if we prefix the env var with <code>AIRFLOW_VAR_</code>. Relevant <a href="https://airflow.apache.org/docs/apache-airflow/stable/concepts.html#variables">Airflow documentation</a>.</p>
<p>In the UI, it is available under <code>Admin>Variables</code>. There you can create edit and delete and env var.</p>
<p>Once set, you can use an airflow variable like below:</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">with</span> DAG<span class="token punctuation">(</span><br /> dag_id<span class="token operator">=</span><span class="token string">"my-little-pony"</span><span class="token punctuation">,</span> schedule_interval<span class="token operator">=</span><span class="token boolean">None</span><span class="token punctuation">,</span> start_date<span class="token operator">=</span>YESTERDAY<br /><span class="token punctuation">)</span> <span class="token keyword">as</span> dag<span class="token punctuation">:</span><br /> head <span class="token operator">=</span> KubernetesPodOperator<span class="token punctuation">(</span><br /> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><br /> cmds<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"python"</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> image<span class="token operator">=</span>Variable<span class="token punctuation">.</span>get<span class="token punctuation">(</span><span class="token string">"pony_builder_image_name"</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment"># accessing variable</span><br /> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><br /> <span class="token punctuation">)</span><br /></code></pre>
<h1 id="method-3---templating-and-macros" tabindex="-1">Method 3 - Templating and macros <a class="direct-link" href="https://blog.meain.io/2021/airflow-template-in-dag/#method-3---templating-and-macros">#</a></h1>
<p>Airflow actually uses Jinja templates for some stuff. It is kinda restricted, but it does provide some nice conveniences.</p>
<p>You can read more about it in the Airflow reference for <a href="https://airflow.apache.org/docs/apache-airflow/stable/concepts.html#jinja-templating">Jinja Templating</a> and <a href="https://airflow.apache.org/docs/apache-airflow/stable/macros-ref.html">macros</a>.</p>
<p>I use this primary for values that are specific to the runtime of a airflow job. Things like <code>run_id</code> or <code>ds</code>(datetime stamp). You can also use this for two other things. You can use this to template out things from the <code>config</code> that gets passed in when you trigger a job from the UI or using the API. The config will be the json object and you can drill down any levels deep with just the <code>.</code>. You can also use this to template variables that I have mentioned earlier using <code>var</code>.</p>
<p>Here is a small example of where I use it.</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">with</span> DAG<span class="token punctuation">(</span><br /> dag_id<span class="token operator">=</span><span class="token string">"my-little-pony"</span><span class="token punctuation">,</span> schedule_interval<span class="token operator">=</span><span class="token boolean">None</span><span class="token punctuation">,</span> start_date<span class="token operator">=</span>YESTERDAY<br /><span class="token punctuation">)</span> <span class="token keyword">as</span> dag<span class="token punctuation">:</span><br /> head <span class="token operator">=</span> KubernetesPodOperator<span class="token punctuation">(</span><br /> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><br /> cmds<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"python"</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> arguments<span class="token operator">=</span><span class="token punctuation">[</span><br /> <span class="token string">"{{ run_id }}"</span><span class="token punctuation">,</span> <span class="token comment"># use of run_id</span><br /> <span class="token string">"{{ dag_run.conf.name }}"</span><span class="token punctuation">,</span> <span class="token comment"># use of value from config</span><br /> <span class="token string">"{{ dag_run.conf.head.type }}"</span><span class="token punctuation">,</span> <span class="token comment"># use of nested config value</span><br /> <span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><br /> <span class="token punctuation">)</span></code></pre>
<h1 id="method-4---from-env-variables" tabindex="-1">Method 4 - From env variables <a class="direct-link" href="https://blog.meain.io/2021/airflow-template-in-dag/#method-4---from-env-variables">#</a></h1>
<p>This is the main reason why I wanted to write this. This is mostly what I wanted to be able to do with my dag. The main reason why I got into this hunt in the first place is that I wanted to template the value of <code>runAsUser</code> etc from a kubernetes secret. I didn't ask for much man, but this lead to me to a relatively long hunt. Either my googling(ducking) skills are not so good or I was just not looking for the right thing. But anyways, I am here and I finally have solution.</p>
<p>This is what I wanted the code to end up looking like.</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">with</span> DAG<span class="token punctuation">(</span><br /> dag_id<span class="token operator">=</span><span class="token string">"my-little-pony"</span><span class="token punctuation">,</span> schedule_interval<span class="token operator">=</span><span class="token boolean">None</span><span class="token punctuation">,</span> start_date<span class="token operator">=</span>YESTERDAY<br /><span class="token punctuation">)</span> <span class="token keyword">as</span> dag<span class="token punctuation">:</span><br /> head <span class="token operator">=</span> KubernetesPodOperator<span class="token punctuation">(</span><br /> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><br /> cmds<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"python"</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> arguments<span class="token operator">=</span><span class="token punctuation">[</span><br /> <span class="token string">"{{ run_id }}"</span><span class="token punctuation">,</span> <span class="token comment"># use of run_id</span><br /> <span class="token string">"{{ dag_run.conf.name }}"</span><span class="token punctuation">,</span> <span class="token comment"># use of value from config</span><br /> <span class="token string">"{{ dag_run.conf.head.type }}"</span><span class="token punctuation">,</span> <span class="token comment"># use of nested config value</span><br /> <span class="token punctuation">]</span><span class="token punctuation">,</span><br /> security_context<span class="token operator">=</span><span class="token punctuation">{</span><br /> <span class="token string">"runAsUser"</span><span class="token punctuation">:</span> <span class="token builtin">int</span><span class="token punctuation">(</span>os<span class="token punctuation">.</span>environ<span class="token punctuation">[</span><span class="token string">"PONY_RUN_AS_USER"</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token string">"runAsGroup"</span><span class="token punctuation">:</span> <span class="token builtin">int</span><span class="token punctuation">(</span>os<span class="token punctuation">.</span>environ<span class="token punctuation">[</span><span class="token string">"PONY_RUN_AS_GROUP"</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><br /> <span class="token punctuation">)</span></code></pre>
<p>So, if you see, things are relatively simple. I just load the env var from <code>PONY_RUN_AS_USER</code> and just use it. I have to convert it to int, but other than that, I just want to load it.</p>
<p>Quick first logical step would be to modify the airflow deployment yaml file to include these things when creating the image. This is mostly taken out of the yaml file that I was using. If you see, I am pulling the secrets from <code>pony-secrets</code> secret in my kube cluster and setting them as env variables.</p>
<pre class="language-yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> apps/v1<br /><span class="token key atrule">kind</span><span class="token punctuation">:</span> Deployment<br /><span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> airflow<br /> <span class="token key atrule">namespace</span><span class="token punctuation">:</span> pony<span class="token punctuation">-</span>builder<br /><span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">replicas</span><span class="token punctuation">:</span> <span class="token number">1</span><br /> <span class="token key atrule">selector</span><span class="token punctuation">:</span><br /> <span class="token key atrule">matchLabels</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> airflow<br /> <span class="token key atrule">template</span><span class="token punctuation">:</span><br /> <span class="token key atrule">metadata</span><span class="token punctuation">:</span><br /> <span class="token key atrule">labels</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> airflow<br /> <span class="token key atrule">spec</span><span class="token punctuation">:</span><br /> <span class="token key atrule">serviceAccountName</span><span class="token punctuation">:</span> airflow<br /> <span class="token key atrule">initContainers</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> <span class="token string">"init"</span><br /> <span class="token key atrule">image</span><span class="token punctuation">:</span> pony<span class="token punctuation">-</span>artifactory<span class="token punctuation">:</span>latest<br /> <span class="token key atrule">imagePullPolicy</span><span class="token punctuation">:</span> Always<br /> <span class="token key atrule">volumeMounts</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> airflow<span class="token punctuation">-</span>configmap<br /> <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /root/airflow/airflow.cfg<br /> <span class="token key atrule">subPath</span><span class="token punctuation">:</span> airflow.cfg<br /> <span class="token key atrule">env</span><span class="token punctuation">:</span> <span class="token important">&envvars</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> PONY_RUN_AS_USER<br /> <span class="token key atrule">valueFrom</span><span class="token punctuation">:</span><br /> <span class="token key atrule">secretKeyRef</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> pony<span class="token punctuation">-</span>secrets<br /> <span class="token key atrule">key</span><span class="token punctuation">:</span> run_as_user<br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> PONY_RUN_AS_GROUP<br /> <span class="token key atrule">valueFrom</span><span class="token punctuation">:</span><br /> <span class="token key atrule">secretKeyRef</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> pony<span class="token punctuation">-</span>secrets<br /> <span class="token key atrule">key</span><span class="token punctuation">:</span> run_as_group<br /> <span class="token key atrule">command</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token string">"bash"</span><br /> <span class="token key atrule">args</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token string">"-cx"</span><br /> <span class="token punctuation">-</span> <span class="token string">"/root/airflow-test-env-init.sh"</span><br /> <span class="token key atrule">containers</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> webserver<br /> <span class="token key atrule">image</span><span class="token punctuation">:</span> pony<span class="token punctuation">-</span>artifactory<span class="token punctuation">:</span>latest<br /> <span class="token key atrule">imagePullPolicy</span><span class="token punctuation">:</span> Always<br /> <span class="token key atrule">ports</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> webserver<br /> <span class="token key atrule">containerPort</span><span class="token punctuation">:</span> <span class="token number">80</span><br /> <span class="token key atrule">args</span><span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token string">"webserver"</span><span class="token punctuation">]</span><br /> <span class="token key atrule">envFrom</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">secretRef</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> airflow<br /> <span class="token key atrule">env</span><span class="token punctuation">:</span> <span class="token important">*envvars</span><br /> <span class="token key atrule">volumeMounts</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> airflow<span class="token punctuation">-</span>configmap<br /> <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /root/airflow/airflow.cfg<br /> <span class="token key atrule">subPath</span><span class="token punctuation">:</span> airflow.cfg<br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> scheduler<br /> <span class="token key atrule">image</span><span class="token punctuation">:</span> pony<span class="token punctuation">-</span>artifactory<span class="token punctuation">:</span>latest<br /> <span class="token key atrule">imagePullPolicy</span><span class="token punctuation">:</span> Always<br /> <span class="token key atrule">envFrom</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">secretRef</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> airflow<br /> <span class="token key atrule">args</span><span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token string">"scheduler"</span><span class="token punctuation">]</span><br /> <span class="token key atrule">env</span><span class="token punctuation">:</span> <span class="token important">*envvars</span><br /> <span class="token key atrule">volumeMounts</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> airflow<span class="token punctuation">-</span>configmap<br /> <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /root/airflow/airflow.cfg<br /> <span class="token key atrule">subPath</span><span class="token punctuation">:</span> airflow.cfg<br /> <span class="token key atrule">volumes</span><span class="token punctuation">:</span><br /> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> airflow<span class="token punctuation">-</span>configmap<br /> <span class="token key atrule">configMap</span><span class="token punctuation">:</span><br /> <span class="token key atrule">name</span><span class="token punctuation">:</span> airflow<span class="token punctuation">-</span>configmap</code></pre>
<p>But, the issue with this is that these variables will not be available in the worker pods. I am not mentioning the final pods that gets launched, but when you launch the dag that I mentioned earlier, Airflow launches another pod which is the one that manages the actual pod. This intermediately pod does not have access to these env variables. Not sure what kind of a decision that was. I don't know if I am missing some kind of a flag or something that would enable this, but this was not available by default.</p>
<p>The job of this intermediate pod is to just evaluate the dag and then create a proper pod. So, if I have to evaluate the pod, I need access to these variables.</p>
<p>What happened here is that the <code>webserver</code> and the <code>scheduler</code> properly parsed the dags, but when this intermediate container that I mentioned tries to parse this dag config it fails saying that these env variables are not available.</p>
<p>What I ended up doing here is to add there to the <code>kubernetes_secrets</code> section in the configmap that gets mounted.</p>
<pre class="language-properties"><code class="language-properties">[kubernetes_secrets]<br /><span class="token key attr-name">PONY_RUN_AS_USER</span> <span class="token punctuation">=</span> <span class="token value attr-value">pony-secrets=run_as_user</span><br /><span class="token key attr-name">PONY_RUN_AS_GROUP</span> <span class="token punctuation">=</span> <span class="token value attr-value">pony-secrets=run_as_group</span></code></pre>
<p>Now with this, I can actually get it in the intermediate pod. And we can actually get the dag running. All good, finally. We can now finally make ponies.</p>
<blockquote>
<p>Btw, this is how we make ponies. It was a well hidden secret, but now you know. It is all powered by Airflow and Kubernetes. The cuteness of gopher should have tipped you off that something was going on.</p>
</blockquote>
<h1 id="bonus" tabindex="-1">Bonus <a class="direct-link" href="https://blog.meain.io/2021/airflow-template-in-dag/#bonus">#</a></h1>
<p>There is only other option. This is actually pretty well documented. If you don't really need the secret values to be used in the dag, but just want to have pass them as env variables, we can just use <code>airflow.contrib.kubernetes.secret</code>. Here is a sample code:</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">from</span> airflow<span class="token punctuation">.</span>contrib<span class="token punctuation">.</span>kubernetes <span class="token keyword">import</span> secret<br /><br />pony_secrets <span class="token operator">=</span> <span class="token punctuation">[</span><br /> secret<span class="token punctuation">.</span>Secret<span class="token punctuation">(</span><br /> deploy_type<span class="token operator">=</span><span class="token string">"env"</span><span class="token punctuation">,</span><br /> deploy_target<span class="token operator">=</span><span class="token string">"PONY_COLOR_MIX"</span><span class="token punctuation">,</span><br /> secret<span class="token operator">=</span><span class="token string">"pony-secrets"</span><span class="token punctuation">,</span><br /> key<span class="token operator">=</span><span class="token string">"color_mix"</span><span class="token punctuation">,</span><br /> <span class="token punctuation">)</span><span class="token punctuation">,</span><br /> secret<span class="token punctuation">.</span>Secret<span class="token punctuation">(</span><br /> deploy_type<span class="token operator">=</span><span class="token string">"env"</span><span class="token punctuation">,</span><br /> deploy_target<span class="token operator">=</span><span class="token string">"PONY_TAIL_CURVE_RADIUS"</span><span class="token punctuation">,</span><br /> secret<span class="token operator">=</span><span class="token string">"pony-secrets"</span><span class="token punctuation">,</span><br /> key<span class="token operator">=</span><span class="token string">"tail_curve_radius"</span><span class="token punctuation">,</span><br /> <span class="token punctuation">)</span><span class="token punctuation">,</span><br /><span class="token punctuation">]</span><br /><br /><span class="token keyword">with</span> DAG<span class="token punctuation">(</span><br /> dag_id<span class="token operator">=</span><span class="token string">"my-little-pony"</span><span class="token punctuation">,</span> schedule_interval<span class="token operator">=</span><span class="token boolean">None</span><span class="token punctuation">,</span> start_date<span class="token operator">=</span>YESTERDAY<br /><span class="token punctuation">)</span> <span class="token keyword">as</span> dag<span class="token punctuation">:</span><br /> head <span class="token operator">=</span> KubernetesPodOperator<span class="token punctuation">(</span><br /> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><br /> cmds<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"python"</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> arguments<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"build-head"</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> secrets<span class="token operator">=</span>pony_secrets<span class="token punctuation">,</span> <span class="token comment"># use of secrets</span><br /> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><br /> <span class="token punctuation">)</span></code></pre>
<p>With that, I have most of what I had in my TIL bucket. Hope that was useful to someone.</p>
Edit everything in Emacs2021-07-26T00:00:00Zhttps://blog.meain.io/2021/edit-everything-in-emacs/<p>Hello everyone, another installment of me going over my setup. This is one of those things which I don't use all that much but wheneve I do, I really am glad that this is a thing.</p>
<p><video src="https://meain.io/blog-videos/videos/emacs-quick-edit.mp4" controls="">Video of what I am talking about</video></p>
<p>Let me first explain the feature that I am talking about. I have my editor(Emacs) setup exactly how I want it to be(for the most part). Here is a sample workflow which I am trying to fix. Let's say I am responding to someone on a Github issue. I start off with typing, half way through I want to write some code and now I could technically write it in the Github UI itself, but that is gonna be a PITA. Why would I do that when I have such a really good setup for me to write stuff in?</p>
<p>Here is what I doing before. I would copy past whatever I wrote to a scratch buffer in Emacs, set the <code>major-mode</code> to <code>markdown-mode</code> and start editing in Emacs. Once I have everything, I copy the entire buffer to clipboard, go hunt for my browser from all the windows, find the input box, focus on it and paste it there.</p>
<p>With that out of the way, let me show you what I did.</p>
<p>I pretty much converted what I would have done direcly into code. The first part is for me to, when I press some key combo, copy text from the current input box and create a new buffer in Emacs with my specific settings and paste the text we have so far into there. This was done in two pieces. The OS level interaction was done using Hammerspoon(I use macOS). The code is something like below.</p>
<pre class="language-lua"><code class="language-lua"><span class="token keyword">local</span> emacs <span class="token operator">=</span> hs<span class="token punctuation">.</span>application<span class="token punctuation">.</span><span class="token function">find</span><span class="token punctuation">(</span>editor<span class="token punctuation">)</span><br />quick_edit_app <span class="token operator">=</span> hs<span class="token punctuation">.</span>window<span class="token punctuation">.</span><span class="token function">focusedWindow</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"cmd"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"a"</span><span class="token punctuation">)</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"cmd"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"c"</span><span class="token punctuation">)</span><br />emacs<span class="token punctuation">:</span><span class="token function">activate</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"alt"</span><span class="token punctuation">,</span> <span class="token string">"shift"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">";"</span><span class="token punctuation">)</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStrokes</span><span class="token punctuation">(</span><span class="token string">"(meain/quick-edit)"</span><span class="token punctuation">)</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"return"</span><span class="token punctuation">)</span></code></pre>
<p>In the first part all I am doing is backing up the name of the "app" that I am currently editing, then do a <kbd>cmd</kbd><kbd>a</kbd> and <kbd>cmd</kbd><kbd>c</kbd> to select eveything and copy it to clipboard. After that I focus the Emacs window and using <kbd>meta</kbd><kbd>shift</kbd><kbd>;</kbd> I call the Emacs function <code>(meain/quick-edit)</code>. Here is what that does.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/quick-edit</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span><br /> <span class="token string">"Util function for use with hammerspoon quick edit functionality."</span><br /> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">qed-buffer-name</span> <span class="token punctuation">(</span><span class="token car">concatenate</span> <span class="token quoted-symbol variable symbol">'string</span><br /> <span class="token string">"qed-"</span><br /> <span class="token punctuation">(</span><span class="token car">substring</span> <span class="token punctuation">(</span><span class="token car">uuid-string</span><span class="token punctuation">)</span><br /> <span class="token number">0</span><br /> <span class="token number">4</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">generate-new-buffer</span> qed-buffer-name<span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">switch-to-buffer</span> qed-buffer-name<span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">evil-paste-after</span> <span class="token number">1</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">gfm-mode</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p>In here, all I do is create a buffer with the name <code>qed-<uuid></code>, switch to it, paste the contents, and enable <code>gfm-mode</code> (gfm = Github Flavoured Markdown). Well, I told you that I literlly just converted what I was doing manually to this.</p>
<p>OK, now to go editing in the buffer...</p>
<p>Edit over, let's get this thing back into the Github issue.</p>
<pre class="language-lua"><code class="language-lua">hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"alt"</span><span class="token punctuation">,</span> <span class="token string">"shift"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">";"</span><span class="token punctuation">)</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStrokes</span><span class="token punctuation">(</span><span class="token string">"(meain/quick-edit-end)"</span><span class="token punctuation">)</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"return"</span><span class="token punctuation">)</span><br />quick_edit_app<span class="token punctuation">:</span><span class="token function">focus</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"cmd"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"a"</span><span class="token punctuation">)</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"cmd"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"v"</span><span class="token punctuation">)</span></code></pre>
<p>OK, again, I call an elisp function <code>meain/quick-edit-end</code> which looks something like this:</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/quick-edit-end</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span><br /> <span class="token string">"Util function to be executed on qed completion."</span><br /> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">mark-whole-buffer</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">call-interactively</span> <span class="token quoted-symbol variable symbol">'kill-ring-save</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">kill-buffer-and-window</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p>All this does is to copy stuff into clipboard and kill the buffer. Once I have this, I just change focus to <code>quick_edit_app</code> which I had previously saved as the app from which I copied. Once I focus on it, we automatically have our focus on the input box in that page. So all I have to do is <kbd>cmd</kbd><kbd>a</kbd> and <kbd>cmd</kbd><kbd>v</kbd> to change what we have in that input field to what we wrote in Emacs.</p>
<p>And there you go, just another simple hack to make life better. Btw, you can probably even combine this with <a href="https://xenodium.com/emacs-utilities-for-your-os/">popup frames</a> and <a href="https://github.com/joostkremers/writeroom-mode">writeroom-mode</a> if that is your jam.</p>
<p>I have added some more things to it. Here is the full code. <a href="https://github.com/meain/dotfiles/blob/25863934a8bcab3e1075cf2f7564b1290b77d14b/emacs/.config/emacs/init.el#L2186">Elisp part</a> and the <a href="https://github.com/meain/dotfiles/blob/25863934a8bcab3e1075cf2f7564b1290b77d14b/hammerspoon/.config/hammerspoon/init.lua#L363">hammerspoon part</a>.</p>
Zooming into Zoom meetings2021-07-31T00:00:00Zhttps://blog.meain.io/2021/zoom-into-zoom/<p>Well, I have been attending a lot of meetings lately. More than I would like to admit. The number of meetings have gone down compared to the start of the pandemic, but it is still a lot. To be frank, sometimes really you have to talk things through instead of just texting people.</p>
<p>OK, with all that aside, let me get to the actual content. I just wanted to show you a quick simple hack that I use to start a meeting and send the link in the current chat that I am on. I am on macOS and use a tool called <a href="http://www.hammerspoon.org/">hammerspoon</a> to help me with this. But to be frank, it is pretty much the idea that I wanted to put out there than the actual implementation.</p>
<p><video src="https://meain.io/blog-videos/videos/zoom-into-zoom.mp4" controls="">Video of what I am talking about</video></p>
<blockquote>
<p>Clipboard is a universal data transfter medium that almost everything supports</p>
</blockquote>
<p>So, here is what I am doing.</p>
<p>First thing to do is to enable a setting in zoom which copies the meeting url when you start a new meeting. In the general tab, make sure "Copy invite link when I starting a meeting" is checked. Now with that out of the way, let us write the hammerspoon part of the code.</p>
<p><img src="https://blog.meain.io/img/zoom-into-zoom.png" alt="Zoom setting image" /></p>
<p>OK, what the code has to do is:</p>
<ul>
<li>Save current window</li>
<li>Open/Focus zoom</li>
<li>Start meeting</li>
<li>Wait for meeting to start (wait for clipboard to have meeting link)</li>
<li>Focus original window</li>
<li>Paste the link</li>
</ul>
<p>These steps can be literally converted to code:</p>
<pre class="language-lua"><code class="language-lua"><span class="token comment">-- Save current window</span><br /><span class="token keyword">local</span> currentApp <span class="token operator">=</span> hs<span class="token punctuation">.</span>window<span class="token punctuation">.</span><span class="token function">focusedWindow</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /><span class="token comment">-- Open/Focus zoom</span><br />hs<span class="token punctuation">.</span>application<span class="token punctuation">.</span><span class="token function">launchOrFocus</span><span class="token punctuation">(</span><span class="token string">"zoom.us"</span><span class="token punctuation">)</span><br /><span class="token comment">-- Start meeting</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"cmd"</span><span class="token punctuation">,</span> <span class="token string">"ctrl"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"v"</span><span class="token punctuation">)</span> <span class="token comment">-- shortcut to start meeting</span><br /><span class="token comment">-- Wait for meeting to start (wait for clipboard to have meeting link)</span><br /><span class="token keyword">local</span> clipChanged <span class="token operator">=</span> <span class="token function">waitTillClipChanges</span><span class="token punctuation">(</span><span class="token number">7</span><span class="token punctuation">)</span> <span class="token comment">-- util func code below</span><br /><span class="token comment">-- Focus original window</span><br />currentApp<span class="token punctuation">:</span><span class="token function">focus</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /><span class="token comment">-- Paste the link</span><br />hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"cmd"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"v"</span><span class="token punctuation">)</span></code></pre>
<p>And there you go, that is all that you have to do. Just another crude hack that makes my life easier. You can find the full code below or in my <a href="https://github.com/meain/dotfiles/blob/25863934a8bcab3e1075cf2f7564b1290b77d14b/hammerspoon/.config/hammerspoon/init.lua#L324">dotfiles</a> repo.</p>
<pre class="language-lua"><code class="language-lua"><span class="token keyword">function</span> <span class="token function">waitTillClipChanges</span><span class="token punctuation">(</span>maxTime<span class="token punctuation">)</span><br /> <span class="token keyword">local</span> initialClip <span class="token operator">=</span> pasteboard<span class="token punctuation">.</span><span class="token function">getContents</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /> <span class="token keyword">local</span> i <span class="token operator">=</span> maxTime<br /> <span class="token keyword">while</span> <span class="token punctuation">(</span>i <span class="token operator">></span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token keyword">do</span><br /> os<span class="token punctuation">.</span><span class="token function">execute</span><span class="token punctuation">(</span><span class="token string">"sleep "</span> <span class="token operator">..</span> <span class="token function">tonumber</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>pasteboard<span class="token punctuation">.</span><span class="token function">getContents</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">~=</span> initialClip<span class="token punctuation">)</span> <span class="token keyword">then</span><br /> <span class="token keyword">return</span> <span class="token keyword">true</span><br /> <span class="token keyword">end</span><br /> i <span class="token operator">=</span> i <span class="token operator">-</span> <span class="token number">1</span><br /> <span class="token keyword">end</span><br /> <span class="token keyword">return</span> <span class="token keyword">false</span><br /><span class="token keyword">end</span><br /><br />hs<span class="token punctuation">.</span>hotkey<span class="token punctuation">.</span><span class="token function">bind</span><span class="token punctuation">(</span><br /> <span class="token punctuation">{</span><span class="token string">"ctrl"</span><span class="token punctuation">,</span> <span class="token string">"alt"</span><span class="token punctuation">,</span> <span class="token string">"shift"</span><span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token string">"z"</span><span class="token punctuation">,</span><br /> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /> <span class="token keyword">local</span> currentApp <span class="token operator">=</span> hs<span class="token punctuation">.</span>window<span class="token punctuation">.</span><span class="token function">focusedWindow</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /> hs<span class="token punctuation">.</span>application<span class="token punctuation">.</span><span class="token function">launchOrFocus</span><span class="token punctuation">(</span><span class="token string">"zoom.us"</span><span class="token punctuation">)</span><br /> hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"cmd"</span><span class="token punctuation">,</span> <span class="token string">"ctrl"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"v"</span><span class="token punctuation">)</span><br /> <span class="token keyword">local</span> clipChanged <span class="token operator">=</span> <span class="token function">waitTillClipChanges</span><span class="token punctuation">(</span><span class="token number">7</span><span class="token punctuation">)</span><br /> <span class="token keyword">if</span> <span class="token keyword">not</span> clipChanged <span class="token keyword">then</span><br /> <span class="token comment">-- in case zoom was not open, redo the thing</span><br /> hs<span class="token punctuation">.</span>application<span class="token punctuation">.</span><span class="token function">launchOrFocus</span><span class="token punctuation">(</span><span class="token string">"zoom.us"</span><span class="token punctuation">)</span><br /> hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"cmd"</span><span class="token punctuation">,</span> <span class="token string">"ctrl"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"v"</span><span class="token punctuation">)</span><br /> <span class="token function">waitTillClipChanges</span><span class="token punctuation">(</span><span class="token number">7</span><span class="token punctuation">)</span><br /> <span class="token keyword">end</span><br /> currentApp<span class="token punctuation">:</span><span class="token function">focus</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /> hs<span class="token punctuation">.</span>eventtap<span class="token punctuation">.</span><span class="token function">keyStroke</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"cmd"</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"v"</span><span class="token punctuation">)</span><br /> <span class="token keyword">end</span><br /><span class="token punctuation">)</span></code></pre>
<p>K. THX. BYE</p>
A better way to think about tests2021-09-08T00:00:00Zhttps://blog.meain.io/2021/better-way-to-think-about-tests/<p>Tests are awesome, but not a lot of people want to write them(myself included at times). This is just a rant type blog and I am probably being really dumb, but I had to get this out there.</p>
<p>I just wanted to talk about how we introduce beginners to the idea of writing tests. When I started off my career, I was told that tests were a way for one to catch bugs in one's code. Even now, in most cases that is how a lot of beginners are being introduced to the idea of writing tests. When we do this, tests comes out as thing that you don't really have to do if you are writing proper code and also as something that will only benefit you in the future. That I think is the main reason why a lot of people don't write tests, because we all know that we write bug free code.</p>
<p>What I suggest instead is to introduce writing tests as a way for you to automate checking if what you wrote is correct. Well, it might seems as if I just reorderd the words. Let me explain.</p>
<p>Let me walk you through a situation. Let us say that every time you make a change, there is a "all knowing" senior dev sitting beside you, looking at your code and telling you if you mess something up and telling you where you messed it up. That would be great, right? Tests are a similar thing, or at least they can be. It is something that you can have constantly running on your code and letting you know if you made a mistake. When you think about it this way, TDD starts to make a lot more sense.</p>
<p>We should not treat tests as something that can help you catch bugs in the future, but rather something that will help you automate how to check if what you wrote actually makes sense as you write it.</p>
<p>Think about it this way. It does not matter if you are writing tests or not; if you are a sane developer, you will most probably try and run the piece of code that you just wrote to see if it does the thing that you think it should do. Why not automate that? That is what tests are for, not just to catch bugs in future. I know a lot of people who write tests just so that they can get the CI to pass, but it should be something that you want to write and not something that you have to write.</p>
<p>Btw, not only will the "all knowing" senior dev help you with issues in your code, they can even help other people with issues in their code if you have a good CI setup.</p>
<blockquote>
<p>All of this only works if you have a good workflow for running tests in the first place</p>
</blockquote>
<p>Tests should not be something that you treat as something that have to do before you push upstream, but rather something that will actively helps you throughout the process of adding in a feature.</p>
<p>So, to sum it all up, here is how I would introduce someone to testing. You write tests so that you can automate checking if what you wrote works. Once you write a piece of code, you should be able to asks the tests to checked if what you wrote is correct. If you write a really broad test, it will help you to fine out that there is an issue, but if you write more find grained tests, it can tell you where exactly things are going wrong.</p>
<p>These kind of tests, along with the regression tests one accumulates over the years should be enough to build out your "all knowing" senior dev for the most part.</p>
<p>Again, I really wanted to stress on the importance of having a really good workflow. If running tests are gonna be harder than or as hard as just running the thing the thing and seeing if it works, my point becomes moot.</p>
<p>That is it for today. Thanks for reading.</p>
Releasing evil-textobj-tree-sitter2021-09-09T00:00:00Zhttps://blog.meain.io/2021/releasing-evil-textobj-tree-sitter/<p><img src="https://meain.io/blog-videos/gifs/evil-textobj-treesitter.gif" alt="demo of evil-textobj-tree-sitter" /></p>
<blockquote>
<p>Link: <a href="https://github.com/meain/evil-textobj-tree-sitter">https://github.com/meain/evil-textobj-tree-sitter</a></p>
</blockquote>
<p>Hey everyone, just wanted to mention about a new package that I released recently. The package name is <code>evil-textobj-tree-sitter</code> and you can find it on my <a href="https://github.com/meain/evil-textobj-tree-sitter">Github</a>.</p>
<p>This package provides textobjects for <a href="https://github.com/emacs-evil/evil">evil-mode</a> using <a href="https://tree-sitter.github.io/">tree-sitter</a>. It helps add textobjects like functions, classes, loops, conditionals etc into evil-mode.</p>
<p>In case you are not familier with how evil-mode or vim works, let me give you a quick walkhrough of the essentials. You might have heard of vim having differnet modes or it being modal. One of those modes is normal mode. In this mode, every keystroke that you do is a command to the editor instead of a direct edit done on the text. This command structure follows a certain grammar in most cases. It usually follows <code><count><action><target></code>.</p>
<p>Let me give you an example. Before that let me introduce you to a few actions and targets.</p>
<ul>
<li>c: change</li>
<li>d: delete</li>
<li>y: yank (copoy)</li>
</ul>
<p><em>There is a log more, just list a some major ones.</em></p>
<p>Now, for targets:</p>
<ul>
<li>w: word</li>
<li>p: paragraph</li>
<li>i": inside double quotes</li>
<li>t.: from here till the next <code>.</code></li>
</ul>
<p>Once you have these, you can combine them.</p>
<ul>
<li>dw: delete word</li>
<li>cw: change word</li>
<li>yw: yank word</li>
<li>dp: delete paragraph</li>
<li>cp: change paragraph</li>
<li>yp: yank paragraph</li>
</ul>
<p>You get the idea right, right?</p>
<p>So, now to what my plugin does. We help add more <code>targets</code> to this grammar. One such example would be <code>function.inner</code>. Let us assume that we map it to the key combo <code>if</code>.</p>
<p>Now you can do:</p>
<ul>
<li>cif: change inside the function</li>
<li>yif: yank inside the function</li>
</ul>
<p>You see how powerful this becomes? Likewise there are other plugins which add actions as well. For example if you have added in a plugin that addes the comment action, you can ask Emacs to just comment the function without having to manually select the function boundary. Sweet, right?</p>
<p>All of this is powered by tree-sitter. tree-sitter at its core is a laugage parser which will give you an ast like thing that you can query on. This is just scratching the surface of what tree-sitter can do. I am hoping to play around with it more in the coming days. I already have few other tree-sitter related things in my configs which I would like to extract out and make into a plugin in the future. Btw, I owe a lot to the maintainers of <a href="https://github.com/nvim-treesitter/nvim-treesitter-textobjects">nvim-treesitter-textobjects</a> as I have copied the tree-sitter queries directly from that project.</p>
<p>Feel free to checkout the package if you are using Emacs :). It should be available in melpa. You can find more information in the <a href="https://github.com/meain/evil-textobj-tree-sitter/blob/master/README.md">README</a>.</p>
Just flip a coin2021-09-10T00:00:00Zhttps://blog.meain.io/2021/just-flip-a-coin/<p>A simple life hack that I use a lot. I just wanted to mention why I always use a coin toss to choose if I can't decide between two options.</p>
<p>There is only two scenarios I am in when I can't decide between two options</p>
<ol>
<li>I know what to pick, but just can't finalize</li>
<li>I genuinely have no clue what to pick</li>
</ol>
<p>In the first case, the only thing that is stopping me on confirming what I want to pick is the expectation that I can always take a little more time and "maybe" come up with a better choice. The coin toss helps to eliminate the amount of time that I have to decide and force me to make a decision. I never really have to go with what the coin picked in this case, but once the coin picks something and if that is not what I want, I know which one I don't want and can pick the other one. Or if it is the one that I wanted to pick, it is like a confirmation from the universe. Either ways, I now have finalized what I want.</p>
<p>In the second case, well I just choose whatever the coin tells me.</p>
<p>And the best part about this is that you don't even have to know which scenario you are in before doing the toss. You will get the urge to change it as soon as the coin picks something if you are in scenario one.</p>
Emacs alternatives for pouplar Neovim plugins2021-09-29T00:00:00Zhttps://blog.meain.io/2021/Emacs-alternatives-for-the-Vim-plugins-you-know-and-love/<p>I was a Neovim user for a long time, but I have switch to using Emacs for quite some time now. About 1.5 years.
The transition for the most part was pretty smooth except for some initial hiccup. I am now in a state where I as productive or even more when compared to Neovim. So I thought I would write about my experience and what all plugins I have found here that helped me replicate my neovim setup.</p>
<blockquote>
<p>You can find both my old neovim config and my new Emacs config at <a href="https://github.com/meain/dotfiles">meain/dotfiles</a></p>
</blockquote>
<p>Let me start of with things that I have found alternatives for.</p>
<table>
<thead>
<tr>
<th>Use</th>
<th>Neovim</th>
<th>Emacs</th>
<th>Note</th>
</tr>
</thead>
<tbody>
<tr>
<td>Show indent lines</td>
<td><a href="https://github.com/Yggdroot/indentLine">Yggdroot/indentLine</a></td>
<td><a href="https://github.com/zk-phi/indent-guide">zk-phi/indent-guide</a></td>
<td>Allows you to conditionally render as well</td>
</tr>
<tr>
<td>Show git changes in buffer</td>
<td><a href="https://github.com/airblade/vim-gitgutter">airblade/vim-gitgutter</a></td>
<td><a href="https://github.com/dgutov/diff-hl">dgutov/diff-hl</a></td>
<td>Emacs version is much more powerful</td>
</tr>
<tr>
<td>Minimal writing env</td>
<td><a href="https://github.com/junegunn/goyo.vim">junegunn/goyo.vim</a></td>
<td><a href="https://github.com/joostkremers/writeroom-mode">joostkremers/writeroom-mode</a></td>
<td></td>
</tr>
<tr>
<td>Highlight only current section</td>
<td><a href="https://github.com/junegunn/limelight.vim">junegunn/limelight.vim</a></td>
<td><a href="https://github.com/larstvei/Focus">larstvei/Focus</a></td>
<td>Emacs versino is more configurable</td>
</tr>
<tr>
<td>Highlight hex or other color codes</td>
<td><a href="https://github.com/RRethy/vim-hexokinase">rrethy/vim-hexokinase</a></td>
<td><a href="http://elpa.gnu.org/packages/rainbow-mode.html">rainbow-mode</a></td>
<td></td>
</tr>
<tr>
<td>Fuzzy matching on stuff</td>
<td><a href="https://github.com/junegunn/fzf.vim">junegunn/fzf.vim</a></td>
<td><a href="https://github.com/raxod502/selectrum">raxod502/selectrum</a> + <a href="https://github.com/raxod502/prescient.el">raxod502/prescient.el</a> + <a href="https://github.com/minad/marginalia">minad/marginalia</a></td>
<td>Insanely more powerful in Emacs</td>
</tr>
<tr>
<td>Markdown preview</td>
<td><a href="https://githubhelp.com/azzsh/vim-instant-markdown">suan/vim-instant-markdown</a></td>
<td><a href="https://github.com/meain/dotfiles/blob/78ce688f2835f0527576ec20a9cb61c44225c267/emacs/.config/emacs/init.el#L2136-L2151">:custom</a></td>
<td>Didn't like any of the Emacs builtin ones</td>
</tr>
<tr>
<td>Move text vertically/horizontally</td>
<td><a href="https://github.com/zirrostig/vim-schlepp">zirrostig/vim-schlepp</a></td>
<td><a href="https://github.com/mkhl/drag-stuff">mkhl/drag-stuff</a></td>
<td>Not as good as Neovim one for horizontal movement</td>
</tr>
<tr>
<td>Run commands async</td>
<td><a href="https://github.com/tpope/vim-dispatch">tpope/vim-dispatch</a></td>
<td>builtin - <code>async-shell-command</code></td>
<td></td>
</tr>
<tr>
<td>Auto set default dir for file</td>
<td><a href="https://github.com/airblade/vim-rooter">airblade/vim-rooter</a></td>
<td><a href="https://github.com/meain/dotfiles/blob/78ce688f2835f0527576ec20a9cb61c44225c267/emacs/.config/emacs/init.el#L2737-L2746">:custom</a></td>
<td></td>
</tr>
<tr>
<td>Auto set tabstop/indent</td>
<td><a href="https://github.com/tpope/vim-sleuth">tpope/vim-sleuth</a></td>
<td><a href="https://github.com/jscheid/dtrt-indent">jscheid/dtrt-indent</a></td>
<td></td>
</tr>
<tr>
<td>Git integration</td>
<td><a href="https://github.com/tpope/vim-fugitive">tpope/vim-fugitive</a></td>
<td><a href="https://github.com/magit/magit">magit/magit</a></td>
<td>Way more powerful</td>
</tr>
<tr>
<td>File symbol tree</td>
<td><a href="https://github.com/liuchengxu/vista.vim">liuchengxu/vista.vim</a></td>
<td><a href="https://github.com/bmag/imenu-list">bmag/imenu-list</a></td>
<td></td>
</tr>
<tr>
<td>Undo tree</td>
<td><a href="https://github.com/mbbill/undotree">mbbill/undotree</a></td>
<td>builtin - <code>undo-tree</code></td>
<td></td>
</tr>
<tr>
<td>File browser</td>
<td><a href="https://github.com/justinmk/vim-dirvish">justinmk/vim-dirvish</a></td>
<td>builtin - <code>dired</code></td>
<td>Way more powerful</td>
</tr>
<tr>
<td>Tree browser</td>
<td><a href="https://github.com/scrooloose/nerdtree">scrooloose/nerdtree</a></td>
<td><a href="https://github.com/jaypei/emacs-neotree">jaypei/emacs-neotree</a></td>
<td>Just used dired</td>
</tr>
<tr>
<td>Surround vim objects with stuff</td>
<td><a href="https://github.com/tpope/vim-surround">tpope/vim-surround</a></td>
<td><a href="https://github.com/emacs-evil/evil-surround">emacs-evil/evil-surround</a></td>
<td></td>
</tr>
<tr>
<td>Auto close brackets</td>
<td><a href="https://github.com/tmsvg/pear-tree">tmsvg/pear-tree</a></td>
<td>builtin - <code>electric-pair</code></td>
<td></td>
</tr>
<tr>
<td>Commenting</td>
<td><a href="https://github.com/tomtom/tcomment_vim">tomtom/tcomment_vim</a></td>
<td><a href="https://github.com/linktohack/evil-commentary">linktohack/evil-commentary</a></td>
<td></td>
</tr>
<tr>
<td>Linting/Checking</td>
<td><a href="https://github.com/w0rp/ale">w0rp/ale</a></td>
<td>builtin - <code>flymake</code></td>
<td></td>
</tr>
<tr>
<td>LSP</td>
<td>builting - <code>lsp</code></td>
<td><a href="https://github.com/joaotavora/eglot">eglot</a></td>
<td></td>
</tr>
<tr>
<td>tree-sitter</td>
<td>builtin - <code>tree-sitter</code></td>
<td><a href="https://githubhelp.com/emacs-tree-sitter/elisp-tree-sitter">emacs-tree-sitter/elisp-tree-sitter</a></td>
<td>Neovim ecosystem is a bit better</td>
</tr>
<tr>
<td>Doc in echo area</td>
<td><a href="https://github.com/Shougo/echodoc.vim">Shougo/echodoc.vim</a></td>
<td>builtin - <code>eldoc</code></td>
<td></td>
</tr>
<tr>
<td>Run tests</td>
<td><a href="https://github.com/janko/vim-test">janko/vim-test</a></td>
<td><a href="https://github.com/meain/dotfiles/blob/78ce688f2835f0527576ec20a9cb61c44225c267/emacs/.config/emacs/init.el#L1526-L1554">:custom</a></td>
<td>Neovim version is much better</td>
</tr>
<tr>
<td>Git commit info</td>
<td><a href="https://github.com/rhysd/git-messenger.vim">rhysd/git-messenger.vim</a></td>
<td><a href="https://github.com/emacsorphanage/git-messenger">emacsorphanage/git-messenger</a></td>
<td>Neovim version is better</td>
</tr>
<tr>
<td>Linting for non-code</td>
<td><a href="https://github.com/davidbeckingsale/writegood.vim">davidbeckingsale/writegood.vim</a></td>
<td><a href="https://github.com/bnbeckwith/writegood-mode">bnbeckwith/writegood-mode</a></td>
<td></td>
</tr>
<tr>
<td>Autocomplete</td>
<td><a href="https://github.com/nvim-lua/completion-nvim">nvim-lua/completion-nvim</a></td>
<td><a href="https://github.com/company-mode/company-mode">company-mode/company-mode</a></td>
<td>Emacs version is a bit more configurable</td>
</tr>
<tr>
<td>Jump to def with regex</td>
<td><a href="https://github.com/pechorin/any-jump.vim">pechorin/any-jump.vim</a></td>
<td><a href="https://github.com/jacktasia/dumb-jump">jacktasia/dumb-jump</a></td>
<td></td>
</tr>
<tr>
<td>Formatter</td>
<td><a href="https://github.com/w0rp/ale">w0rp/ale</a></td>
<td><a href="https://github.com/lassik/emacs-format-all-the-code">lassik/emacs-format-all-the-code</a></td>
<td></td>
</tr>
</tbody>
</table>
<p>Beyond this, I have ported most of my plugins and themes as well to Emacs. I just have not separated them out and published yet. It was a relatively easy process to port them thought. Elisp is really good language to work with.</p>
<p>Now to things that Neovim cannot do (as far as I know)</p>
<ul>
<li><a href="https://github.com/clemera/dired-git-info">dired-git-info</a> Provides you with github like info in file browser thingy</li>
<li><a href="https://github.com/bbatsov/projectile">projectile</a> Project based workflows</li>
<li><a href="https://github.com/magit/forge">forge</a> Interactive with Github and other git hosting providers</li>
<li><a href="https://www.emacswiki.org/emacs/WinnerMode">winner</a> Retain/Undo-Redo window configs</li>
<li><a href="https://notmuchmail.org/notmuch-emacs/">notmuch</a> Email</li>
<li><a href="https://github.com/skeeto/elfeed">elfeed</a> RSS</li>
<li><a href="https://www.emacswiki.org/emacs/TrampMode">tramp</a> Rmote editing (not that reliable though)</li>
<li><a href="https://github.com/Silex/docker.el">docker</a> Docker interface from Emacs</li>
<li><a href="https://github.com/kubernetes-el/kubernetes-el">kubernetes</a> Kubernetes interface from Emacs</li>
</ul>
<p>None of this is to ask you to switch to Emacs or anything. It is more for people who are new to Emacs and are Vim converts trying to figure thing out.</p>
Releasing gh-issues-to-rss2021-10-13T00:00:00Zhttps://blog.meain.io/2021/releasing-gh-issues-to-rss/<blockquote>
<p>Project Url: <a href="https://github.com/meain/gh-issues-to-rss">https://github.com/meain/gh-issues-to-rss</a></p>
</blockquote>
<p>Yet another package release coming your way. This is more of a scratch my own itch kinda entry. There are lot of projects for which I would like to know when things happen. As of now, there are two options in Github to do this:</p>
<ol>
<li>Subscribe to just releases releases and be late to the party</li>
<li>Subscribe to all events and be tired of notifications</li>
</ol>
<p>Well, both of them suck. This is my way of solving that issue. This is a simple go server which gives you rss feeds for issues or PRs. For example, in a lot of cases, I would just want to get one entry every time a new PR is open. A really good usecase for this is for <a href="https://github.com/melpa/melpa">melpa/melpa</a> or <a href="https://github.com/matrix-org/matrix-doc">matrix-org/matrix-doc</a>. By subscribing to PR open events via rss, I get an entry in my rss feed reader every time a new PR is open. There are also a lot of other projects for which I just follow PR open events or issue close events and then if necessary I can just subscribe to a specific thread to get a notification every time someone comments on it.</p>
<p>This has been really helpful for me to reduce clutter as well as move my non important Github notification to a passive medium (rss). I am no more constantly bombarded with notification on Github.</p>
<p>Enough intro, let me get to how to use it. I advice you to host this if possible, but if you want you can just used the ~heroku~ fly.dev deployment that I have setup. There is a 60 request in hour limit on Github API, so if a lot of people are using it, you might not get your response. Btw, I have made a simple UI as well which can help you with getting the rss feed url with proper flags. Hopefully that comes in handy to anyone using this.</p>
<p>You can connect to the fly.dev version at <a href="https://gh-issues-to-rss.fly.dev/">https://gh-issues-to-rss.fly.dev</a>
Let me just give you a simple intro on how to use it.</p>
<p>If you want to subscribe to melpa/melpa on Github, you can use something like:<br />
<a href="https://gh-issues-to-rss.fly.dev/melpa/melpa">https://gh-issues-to-rss.fly.dev/melpa/melpa</a></p>
<blockquote>
<p>This will subscribe you to issue open/close and pr open/close events.</p>
</blockquote>
<p>You can limit what kind of events you get a notification for by using <code>m</code> query params</p>
<ul>
<li>ic: issue-closed</li>
<li>io: issue-open</li>
<li>pc: pr-closed</li>
<li>po: pr-open</li>
</ul>
<p>For example, as I mentioned before, if you just need PR open events, you can use something like:<br />
<a href="https://gh-issues-to-rss.fly.dev/melpa/melpa?m=po">https://gh-issues-to-rss.fly.dev/melpa/melpa?m=po</a></p>
<p>You can add multiple items as well, for both pr and issue close events, you can use:<br />
<a href="https://gh-issues-to-rss.fly.dev/melpa/melpa?m=pc&m=ic">https://gh-issues-to-rss.fly.dev/melpa/melpa?m=pc&m=ic</a></p>
<p>You also have the ability to filter by labels. For example, if you want to have an rss feed for <code>good-first-issue</code> items, you can use:<br />
<a href="https://gh-issues-to-rss.fly.dev/melpa/melpa?l=good-first-issue&m=io">https://gh-issues-to-rss.fly.dev/melpa/melpa?l=good-first-issue&m=io</a><br />
The above one will get you open events for good-first-issue labelled items.</p>
<p>This has been really helpful for me in reducing clutter and at the same time staing up to date with things.
So I thought I would release this as it might end up being useful for someone else as well.</p>
Intelligent snippets using tree-sitter2021-10-16T00:00:00Zhttps://blog.meain.io/2021/intelligent-snippets-treesitter/<p>I found myself writing a lot more go these days. So far I like the language even thought it seems way too simple at times. That said, one thing that I found a bit annoying was how I had to write all those default values when we have to return an error. I really liked how Rust handed errors, but yeah.</p>
<p>In this article I wanted to introduce a snippet that I use which will automatically fill in that <code>if err != nill</code> thing in your go program with all the proper return values for you. I actually got this idea from <a href="https://twitter.com/TeejDeVries">TJ DeVries</a>, a Neovim core contributor.</p>
<p>Let me first show you the code and then we can go over it.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/go-default-returns</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">type</span></span><span class="token punctuation">)</span></span><br /> <span class="token string">"Making it a function instead of an alist so that we can handle unknown <span class="token argument">TYPE</span>."</span><br /> <span class="token punctuation">(</span><span class="token car">pcase</span> type<br /> <span class="token punctuation">(</span><span class="token string">"error"</span> <span class="token string">"err"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token string">"string"</span> <span class="token string">"\"\""</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token string">"rune"</span> <span class="token string">"0"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token string">"int"</span> <span class="token string">"0"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token string">"float64"</span> <span class="token string">"0.0"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token string">"bool"</span> <span class="token string">"false"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token string">"chan"</span> <span class="token string">"nil"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">pred</span> <span class="token punctuation">(</span><span class="token car">string-prefix-p</span> <span class="token string">"<-"</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token string">"nil"</span><span class="token punctuation">)</span> <span class="token comment">; channels</span><br /> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">pred</span> <span class="token punctuation">(</span><span class="token car">string-prefix-p</span> <span class="token string">"["</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token string">"nil"</span><span class="token punctuation">)</span> <span class="token comment">; arrays</span><br /> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">pred</span> <span class="token punctuation">(</span><span class="token car">string-match</span> <span class="token string">" "</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token boolean">nil</span><span class="token punctuation">)</span> <span class="token comment">; for situations with return name</span><br /> <span class="token punctuation">(</span><span class="token car">_</span> <span class="token punctuation">(</span><span class="token keyword">concat</span> type <span class="token string">"{}"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/go-return-string</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span><br /> <span class="token string">"Get return string for go by looking up the return type of current func."</span><br /> <span class="token punctuation">(</span><span class="token keyword">let*</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">func-node</span> <span class="token punctuation">(</span><span class="token car">tree-sitter-node-at-point</span> <span class="token quoted-symbol variable symbol">'function_declaration</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">return-node</span> <span class="token punctuation">(</span><span class="token car">tsc-get-child-by-field</span> func-node '<span class="token lisp-property property">:result</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token comment">;; remove extra whitespace if nothing at end</span><br /> <span class="token punctuation">(</span><span class="token car">replace-regexp-in-string</span> <span class="token string">" $"</span> <span class="token string">""</span><br /> <span class="token punctuation">(</span><span class="token keyword">concat</span> <span class="token string">"return "</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> return-node<br /> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">return-node-type</span> <span class="token punctuation">(</span><span class="token car">tsc-node-type</span> return-node<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">return-node-text</span> <span class="token punctuation">(</span><span class="token car">tsc-node-text</span> return-node<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">pcase</span> return-node-type<br /> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'parameter_list</span> <span class="token punctuation">(</span><span class="token car">string-join</span> <br /> <span class="token punctuation">(</span><span class="token car">remove-if</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token car">equal</span> <span class="token boolean">nil</span> x<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token quoted-symbol variable symbol">'meain/go-default-returns</span><br /> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token quoted-symbol variable symbol">'string-trim</span><br /> <span class="token punctuation">(</span><span class="token car">split-string</span> <span class="token punctuation">(</span><span class="token car">string-trim</span> return-node-text <span class="token string">"("</span> <span class="token string">")"</span><span class="token punctuation">)</span> <span class="token string">","</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token string">", "</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">_</span> <span class="token punctuation">(</span><span class="token car">meain/go-default-returns</span> return-node-text<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<blockquote>
<p>You can find it in my <a href="https://github.com/meain/dotfiles/blob/2cd46a6c9d90570ae8b4ff3e494a90ba42e910e1/emacs/.config/emacs/init.el#L516">dotfiles</a></p>
</blockquote>
<p>Here is what the code does, the first function <code>meain/go-default-returns</code> is a lookup table mapping each type to its default return value. The second function <code>meain/go-return-string</code> is what does the bulk of the work.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="highlight-line"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/go-return-string</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span></span><br /><span class="highlight-line"> <span class="token string">"Get return string for go by looking up the return type of current func."</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token keyword">let*</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">func-node</span> <span class="token punctuation">(</span><span class="token car">tree-sitter-node-at-point</span> <span class="token quoted-symbol variable symbol">'function_declaration</span><span class="token punctuation">)</span><span class="token punctuation">)</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token car">return-node</span> <span class="token punctuation">(</span><span class="token car">tsc-get-child-by-field</span> func-node '<span class="token lisp-property property">:result</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></mark><br /><span class="highlight-line"> <span class="token comment">;; remove extra whitespace if nothing at end</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">replace-regexp-in-string</span> <span class="token string">" $"</span> <span class="token string">""</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">concat</span> <span class="token string">"return "</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">if</span> return-node</span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">return-node-type</span> <span class="token punctuation">(</span><span class="token car">tsc-node-type</span> return-node<span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">return-node-text</span> <span class="token punctuation">(</span><span class="token car">tsc-node-text</span> return-node<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">pcase</span> return-node-type</span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'parameter_list</span> <span class="token punctuation">(</span><span class="token car">string-join</span> </span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">remove-if</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token car">equal</span> <span class="token boolean">nil</span> x<span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token quoted-symbol variable symbol">'meain/go-default-returns</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token quoted-symbol variable symbol">'string-trim</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">split-string</span> <span class="token punctuation">(</span><span class="token car">string-trim</span> return-node-text <span class="token string">"("</span> <span class="token string">")"</span><span class="token punctuation">)</span> <span class="token string">","</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token string">", "</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">_</span> <span class="token punctuation">(</span><span class="token car">meain/go-default-returns</span> return-node-text<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span></code></pre>
<p>The 2nd and 3rd lines (the ones highlighted above) does the initial tree-sitter query. It first find out the function that we are in, and then looks up the result section by its label. You can check the tree sitter tree using the <a href="https://tree-sitter.github.io/tree-sitter/playground">online playground</a>. Here is what a sample go code will look like.</p>
<p><img src="https://blog.meain.io/img/tree-sitter-playground.png" alt="Screenshot of tree sitter playground" /></p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="highlight-line"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/go-return-string</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span></span><br /><span class="highlight-line"> <span class="token string">"Get return string for go by looking up the return type of current func."</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">let*</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">func-node</span> <span class="token punctuation">(</span><span class="token car">tree-sitter-node-at-point</span> <span class="token quoted-symbol variable symbol">'function_declaration</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">return-node</span> <span class="token punctuation">(</span><span class="token car">tsc-get-child-by-field</span> func-node '<span class="token lisp-property property">:result</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token comment">;; remove extra whitespace if nothing at end</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">replace-regexp-in-string</span> <span class="token string">" $"</span> <span class="token string">""</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">concat</span> <span class="token string">"return "</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token keyword">if</span> return-node</mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">return-node-type</span> <span class="token punctuation">(</span><span class="token car">tsc-node-type</span> return-node<span class="token punctuation">)</span><span class="token punctuation">)</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token car">return-node-text</span> <span class="token punctuation">(</span><span class="token car">tsc-node-text</span> return-node<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></mark><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">pcase</span> return-node-type</span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'parameter_list</span> <span class="token punctuation">(</span><span class="token car">string-join</span> </span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">remove-if</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token car">equal</span> <span class="token boolean">nil</span> x<span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token quoted-symbol variable symbol">'meain/go-default-returns</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token quoted-symbol variable symbol">'string-trim</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">split-string</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">string-trim</span> return-node-text <span class="token string">"("</span> <span class="token string">")"</span><span class="token punctuation">)</span> <span class="token string">","</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token string">", "</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">_</span> <span class="token punctuation">(</span><span class="token car">meain/go-default-returns</span> return-node-text<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span></code></pre>
<p>Now, if we are actually able to find a node, we continue and query tree-sitter to find out what kind of node it is and the actual content. We need to get the node type so that we can determine if it is just a single return or if there are multiple items to return.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="highlight-line"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/go-return-string</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span></span><br /><span class="highlight-line"> <span class="token string">"Get return string for go by looking up the return type of current func."</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">let*</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">func-node</span> <span class="token punctuation">(</span><span class="token car">tree-sitter-node-at-point</span> <span class="token quoted-symbol variable symbol">'function_declaration</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">return-node</span> <span class="token punctuation">(</span><span class="token car">tsc-get-child-by-field</span> func-node '<span class="token lisp-property property">:result</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token comment">;; remove extra whitespace if nothing at end</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">replace-regexp-in-string</span> <span class="token string">" $"</span> <span class="token string">""</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">concat</span> <span class="token string">"return "</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">if</span> return-node</span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">return-node-type</span> <span class="token punctuation">(</span><span class="token car">tsc-node-type</span> return-node<span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">(</span><span class="token car">return-node-text</span> <span class="token punctuation">(</span><span class="token car">tsc-node-text</span> return-node<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token car">pcase</span> return-node-type</mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'parameter_list</span> <span class="token punctuation">(</span><span class="token car">string-join</span> </mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token car">remove-if</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token car">equal</span> <span class="token boolean">nil</span> x<span class="token punctuation">)</span><span class="token punctuation">)</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token quoted-symbol variable symbol">'meain/go-default-returns</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token quoted-symbol variable symbol">'string-trim</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token car">split-string</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token car">string-trim</span> return-node-text <span class="token string">"("</span> <span class="token string">")"</span><span class="token punctuation">)</span> <span class="token string">","</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token string">", "</span><span class="token punctuation">)</span><span class="token punctuation">)</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">(</span><span class="token car">_</span> <span class="token punctuation">(</span><span class="token car">meain/go-default-returns</span> return-node-text<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></mark></code></pre>
<p>After that, we do a conditional and separately handle single vs multi entry ones. If there are multiple entries, the node-type will be <code>parameter_list</code> and we handle that in the first section. Let's get to that later. If it is not that, then we have the simple case of a single return statement for which we can use the above function we wrote to get the return, which is then prepended with "return " to get the final output.</p>
<p>In the case where we have multiple items, you have to read it inside out.</p>
<ul>
<li>strip "(" and ")"</li>
<li>split string by ","</li>
<li>trim any spaces</li>
<li>map the function we wrote to get default over the values</li>
<li>join them up with ", "</li>
</ul>
<p>And we are good to go. Now that we have this, chuck in a <code>if err != nill {}</code> around it and wire up to your favorite snippet expansion library and you are good to go. I personally use <a href="https://github.com/ymarco/auto-activating-snippets">auto-activating-snippets</a> and you can find my setup <a href="https://github.com/meain/dotfiles/blob/2cd46a6c9d90570ae8b4ff3e494a90ba42e910e1/emacs/.config/emacs/init.el#L647">here</a>.</p>
<p>Hopefully you found that useful ;)</p>
Introduction to nix2021-11-06T00:00:00Zhttps://blog.meain.io/2021/intro-to-nix/<p>I have recently been playing around with <a href="https://nixos.org/">nix</a> a lot. It is a really interesting piece of technology. I haven't figured out most of it, but I have already started looking down on people who don't use it. But yeah, in all seriousness, it seems like a really interesting project.</p>
<p>Being a beginner I thought I would write a beginner ish tutorial on nix. Here is what nix is if you have not heard of it.</p>
<h2 id="what-is-nix%3F" tabindex="-1">What is nix? <a class="direct-link" href="https://blog.meain.io/2021/intro-to-nix/#what-is-nix%3F">#</a></h2>
<p>Let us assume that you are a developer. I think I will go for a "Javascript" developer since that is what most people these days are anyways. And let us assume that you use <code>npm</code> for managing packages.</p>
<p>How would you go about installing a dependency?</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">npm</span> <span class="token function">install</span> is-thirteen <span class="token comment"># I have proundly contributed one commit to this mess</span></code></pre>
<p>OK, do you know what that does to your repo? (No you don't, because you only think about yourself). Let me show you.</p>
<p>First thing that it does is modify your <code>package.json</code> file. <code>is-thirteen</code> gets added to the dependencies list.</p>
<pre class="language-diff"><code class="language-diff">diff --git a/package.json b/package.json<br />index 6ef6e28..d059850 100644<br /><span class="token coord">--- a/package.json</span><br /><span class="token coord">+++ b/package.json</span><br /><span class="token coord">@@ -8,5 +8,7 @@</span><br /><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> },<br /></span><span class="token prefix unchanged"> </span><span class="token line"> "author": "Abin Simon <mail@meain.io>",<br /></span><span class="token prefix unchanged"> </span><span class="token line"> "license": "MIT",<br /></span></span><span class="token deleted-sign deleted"><span class="token prefix deleted">-</span><span class="token line"> "dependencies": {}<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> "dependencies": {<br /></span><span class="token prefix inserted">+</span><span class="token line"> "is-thirteen": "^2.0.0"<br /></span><span class="token prefix inserted">+</span><span class="token line"> }<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">}<br /></span></span></code></pre>
<p>What this specifies is more or less "we need a version of <code>is-thirteen</code> which is above <code>2.0.0</code> and below <code>3.0.0</code>". While that is useful, one thing that we all have learnt is that we cannot relay on people to bump versions properly. So we need to say what exactly is the version that we are depending on. That is where the <code>package-lock.json</code> file comes in.</p>
<pre class="language-diff"><code class="language-diff">diff --git a/package-lock.json b/package-lock.json<br />index e5ab36e..e293cdd 100644<br /><span class="token coord">--- a/package-lock.json</span><br /><span class="token coord">+++ b/package-lock.json</span><br /><span class="token coord">@@ -1,5 +1,21 @@</span><br /><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">{<br /></span><span class="token prefix unchanged"> </span><span class="token line"> "name": "nixblog",<br /></span><span class="token prefix unchanged"> </span><span class="token line"> "version": "1.0.0",<br /></span></span><span class="token deleted-sign deleted"><span class="token prefix deleted">-</span><span class="token line"> "lockfileVersion": 1<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> "lockfileVersion": 1,<br /></span><span class="token prefix inserted">+</span><span class="token line"> "requires": true,<br /></span><span class="token prefix inserted">+</span><span class="token line"> "dependencies": {<br /></span><span class="token prefix inserted">+</span><span class="token line"> "is-thirteen": {<br /></span><span class="token prefix inserted">+</span><span class="token line"> "version": "2.0.0",<br /></span><span class="token prefix inserted">+</span><span class="token line"> "resolved": "https://registry.npmjs.org/is-thirteen/-/is-thirteen-2.0.0.tgz",<br /></span><span class="token prefix inserted">+</span><span class="token line"> "integrity": "sha1-otvQ9at+EKTQGG6aCyQmMiTT+bE=",<br /></span><span class="token prefix inserted">+</span><span class="token line"> "requires": {<br /></span><span class="token prefix inserted">+</span><span class="token line"> "noop3": "^13.7.2"<br /></span><span class="token prefix inserted">+</span><span class="token line"> }<br /></span><span class="token prefix inserted">+</span><span class="token line"> },<br /></span><span class="token prefix inserted">+</span><span class="token line"> "noop3": {<br /></span><span class="token prefix inserted">+</span><span class="token line"> "version": "13.8.1",<br /></span><span class="token prefix inserted">+</span><span class="token line"> "resolved": "https://registry.npmjs.org/noop3/-/noop3-13.8.1.tgz",<br /></span><span class="token prefix inserted">+</span><span class="token line"> "integrity": "sha1-CuZBS21947bYUFXNKpIMGg1hrW4="<br /></span><span class="token prefix inserted">+</span><span class="token line"> }<br /></span><span class="token prefix inserted">+</span><span class="token line"> }<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">}</span></span></code></pre>
<p>If you see above, we have a specific version of <code>is-thirteen</code>, ie <code>2.0.0</code>. We also do this for any of the dependencies of <code>is-thirteen</code>. We pin the dependency <code>noop3</code> to <code>13.8.1</code>. This gets done all the way down the stack. What this means is that we can know exactly what version of everything we were using. But if you look into it, the dependency pinning stops at <code>noop3</code>. But what about the <code>node</code> version we are using? Shouldn't we pin that? What about the compiler used to build <code>node</code>? What about the kernel? What about any extra env variables/flags used while building the kernel?</p>
<p>That is exactly where nix comes in, it lets you pin everything all the way down. At least that is what the idea of nix is in general. There is a lot of advantages of doing this of which only one is that your setup is reproducible. To be frank, I got into nix just because I wanted to make sure I have a declarative definition of the packages that I install on my mac, but now I am too deep into this.</p>
<p>When talking about nix, you will primarily come across 3 different things, Nix, nixpkgs and NixOS.</p>
<p>Nix is the language in which all of the nix related stuff is written in. This is also the name of the package manager. <a href="https://github.com/NixOS/nixpkgs"><code>nixpkgs</code></a> is the primary repository for all packages. You can think of this as <code>npm.org</code> or debian/arch repositories. And finally NixOS is a complete operating system based on Linux which is built on the idea of nix. Your entire operating system in this case is declarative.</p>
<h2 id="how-do-i-use-nix%3F" tabindex="-1">How do I use nix? <a class="direct-link" href="https://blog.meain.io/2021/intro-to-nix/#how-do-i-use-nix%3F">#</a></h2>
<p>Well, glad you asked. First of all, install nix. You can do this on a mac or linux machine.</p>
<p><a href="https://nixos.org/download.html#download-nix">https://nixos.org/download.html#download-nix</a></p>
<p>This will install the nix package manager (<em>not the NixOS operating system</em>) on your system. Once you do that, you are ready to have fun with nix.</p>
<p>Let's first try working on a simple project using nix. How about we build a "Goodbye World!" program in Rust? You know it is fancy because we are using Rust ;)</p>
<p>OK, let's write the Rust program. You can drop this in <code>main.rs</code>.</p>
<pre class="language-rust"><code class="language-rust"><span class="token keyword">fn</span> <span class="token function-definition function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token macro property">println!</span><span class="token punctuation">(</span><span class="token string">"Goodbye World!"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Now, we have to compile this. And we do:</p>
<pre class="language-shell"><code class="language-shell">$ rustc main.rs<br />zsh: <span class="token builtin class-name">command</span> not found: rustc</code></pre>
<p>Ohh woopsie, we don't have rustc. Let's use nix to fix that. The simplest thing that you can do is to start a new shell with <code>rustc</code> in it. You can do that with:</p>
<pre class="language-shell"><code class="language-shell">$ nix-shell <span class="token parameter variable">-p</span> rustc</code></pre>
<p>And now if we you run it, it all works. ;)
Well, while that was cool, I could have done that with any other package manager.</p>
<p><em>I know, just wait. Give me some time to explain why nix is better. :D</em></p>
<p>What <code>nix-shell</code> does is that it puts you in a new shell which has rustc available. <code>exit</code> from the shell and see if <code>rustc</code> is still available?</p>
<p>No, right?</p>
<p>Let's get back into that nix shell with the same command and see where <code>rustc</code> comes from.</p>
<pre class="language-shell"><code class="language-shell">$ nix-shell <span class="token parameter variable">-p</span> rustc<br />$ <span class="token function">which</span> rustc<br />/nix/store/vqksgxrd1p091mnvz2bixnr8ylsyima1-rustc-1.52.1/bin/rustc</code></pre>
<p><em>Not where you were expecting it to be from, I guess.</em></p>
<p>Let me tell you what is with that weird path. Everything that you install with nix is installed to <code>/nix/store</code> and just made available for you. So what happened when you did <code>nix-shell -p rust</code> is that <code>rustc</code> was installed to the nix store and made available via the <code>$PATH</code> variable for you.</p>
<p><em>Check what the <code>$PATH</code> variable contains inside the shell.</em></p>
<p>That was a lot of fun tricks, but let's make this thing a bit more declarative. Plus it would be a pain if I had to type out that command if I had multiple dependencies. That is wheres <code>shell.nix</code> comes in.</p>
<p>Drop this in a file called <code>shell.nix</code> in the root of your project.</p>
<pre class="language-nix"><code class="language-nix"><span class="token keyword">with</span> <span class="token punctuation">(</span><span class="token function">import</span> <span class="token operator"><</span>nixpkgs<span class="token operator">></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br />mkShell <span class="token punctuation">{</span><br /> buildInputs <span class="token operator">=</span> <span class="token punctuation">[</span><br /> rustc<br /> <span class="token punctuation">]</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Now, whenever you wanna work on this project, just navigate to the directory and do <code>nix-shell</code>. Nix will see this file and automatically load the env for you. If you are interested, there is a tool called <code>direnv</code> which will automatically "start up" this env for you as soon as you cd into that directory.</p>
<p>Let me explain what that code does thought. Not much, but here is how I that code converts to python:</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">import</span> nixpkgs<br /><br />make_shell<span class="token punctuation">(</span>packages<span class="token operator">=</span><span class="token punctuation">[</span>nixpkgs<span class="token punctuation">.</span>rustc<span class="token punctuation">]</span><span class="token punctuation">)</span></code></pre>
<p>One thing to note here is that we are importing nixpkgs. What happens here is that you are importing whatever nixpkgs version you have in your local system, but if you wanted to pin everything, you can just specify an exact version there and you will be able to trace back to everything that resulted in building the <code>rustc</code> that you use.</p>
<p>Oh, I actually forgot to show you that this thing works now.</p>
<pre class="language-shell"><code class="language-shell">$ rustc main.rs<br />$ ./main<br />Goodbye World<span class="token operator">!</span></code></pre>
<p>You can actually do a lot more than just specify the packages that you need there btw. You can specify the env variables, or some specific aliases in the new shell, or even start up some services or...</p>
<p>Checkout <a href="http://ghedam.at/15978/an-introduction-to-nix-shell">http://ghedam.at/15978/an-introduction-to-nix-shell</a> to learn a bit more about nix shell.</p>
<p>Wait, but I want to build my own package with nix.</p>
<p><em>A wild nix file shows up. This one is called <code>default.nix</code></em></p>
<p><code>default.nix</code> is what you would use for packaging your own application.</p>
<pre class="language-nix"><code class="language-nix"><span class="token punctuation">{</span> nixpkgs <span class="token operator">?</span> <span class="token function">import</span> <span class="token operator"><</span>nixpkgs<span class="token operator">></span> <span class="token punctuation">{</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">:</span><br /><br />nixpkgs<span class="token punctuation">.</span>stdenv<span class="token punctuation">.</span>mkDerivation <span class="token punctuation">{</span><br /> name <span class="token operator">=</span> <span class="token string">"goodbye"</span><span class="token punctuation">;</span><br /> buildInputs <span class="token operator">=</span> <span class="token punctuation">[</span> nixpkgs<span class="token punctuation">.</span>rustc <span class="token punctuation">]</span><span class="token punctuation">;</span><br /> src <span class="token operator">=</span> <span class="token url">./.</span><span class="token punctuation">;</span><br /><br /> buildPhase <span class="token operator">=</span> <span class="token string">''<br /> rustc main.rs<br /> ''</span><span class="token punctuation">;</span><br /><br /> installPhase <span class="token operator">=</span> <span class="token string">''<br /> mkdir -p $out<br /> mv main $out/goodbye<br /> ''</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Let me explain what the code does. Wait, let me convert it to python first.</p>
<pre class="language-python"><code class="language-python"><span class="token keyword">class</span> <span class="token class-name">Goodbye</span><span class="token punctuation">(</span>mkDerivation<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> self<span class="token punctuation">.</span>name <span class="token operator">=</span> <span class="token string">"goodbye"</span><br /> self<span class="token punctuation">.</span>src <span class="token operator">=</span> <span class="token string">"./."</span> <span class="token comment"># current directory</span><br /><br /> <span class="token keyword">def</span> <span class="token function">__init__</span><span class="token punctuation">(</span>self<span class="token punctuation">,</span> nixpkgs<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> self<span class="token punctuation">.</span>dependencies <span class="token operator">=</span> <span class="token punctuation">[</span>nixpkgs<span class="token punctuation">.</span>rustc<span class="token punctuation">]</span><br /> <br /> <span class="token keyword">def</span> <span class="token function">build</span><span class="token punctuation">(</span>self<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> runShellCommand<span class="token punctuation">(</span><span class="token triple-quoted-string string">'''<br /> rustc main.rs<br /> '''</span><span class="token punctuation">)</span><br /><br /> <span class="token keyword">def</span> <span class="token function">install</span><span class="token punctuation">(</span>self<span class="token punctuation">)</span><span class="token punctuation">:</span><br /> runShellCommand<span class="token punctuation">(</span><span class="token triple-quoted-string string">'''<br /> mkdir -p $out<br /> mv main $out/goodbye<br /> '''</span><span class="token punctuation">)</span></code></pre>
<p>This is kinda what it translates to. The entire file in total is a function which takes <code>nixpkgs</code> (<em>This part did not translate well into python</em>) as an argument and creates a so called <code>derivation</code>. A <code>derivation</code> is essentially a set of instructions on how to build something, in this case our goodbye program.</p>
<p>If you check the code, we actually define a few things,</p>
<ul>
<li>name: name of the package</li>
<li>src: source for building a package (this can be a git url etc...)</li>
<li>buildInputs: all the deps for building the package</li>
<li>buildPhase: how to build the project</li>
<li>installPhase: how to install the project</li>
</ul>
<p><em>The whole deal with <code>$out</code> is a bit beyond this blog, but essentially we are specifying what all binaries are to be "installed".</em></p>
<p>Now that we have all of this defined, you can call <code>nix-build</code> in your project and you will get a <code>result</code> directory. This is where we take the <code>derivation</code> and build out the actual thing. If you check where the <code>result</code> directory points, you will again see that nix store path. As I said, everything goes into the nix store and all that you see is just symlinks. You can now run the app from the results directory.</p>
<pre class="language-shell"><code class="language-shell">$ nix-build<br />this derivation will be built:<br /> /nix/store/c48jmny69bp3fyai34mbjirk3f9spxw8-env.drv<br />building <span class="token string">'/nix/store/c48jmny69bp3fyai34mbjirk3f9spxw8-env.drv'</span><span class="token punctuation">..</span>.<br />unpacking sources<br />unpacking <span class="token builtin class-name">source</span> archive /nix/store/5da7rd7iadk2dxj6w7nij615mvdx9sfv-rustgoodbye<br /><span class="token builtin class-name">source</span> root is rustgoodbye<br />patching sources<br />configuring<br />no configure script, doing nothing<br />building<br />installing<br />post-installation fixup<br />shrinking RPATHs of ELF executables and libraries <span class="token keyword">in</span> /nix/store/4mlqz3gx1n09gbcn95jap6rixpcvzdjd-env<br />shrinking /nix/store/4mlqz3gx1n09gbcn95jap6rixpcvzdjd-env/goodbye<br />strip is /nix/store/5ddb4j8z84p6sjphr0kh6cbq5jd12ncs-binutils-2.35.1/bin/strip<br />patching script interpreter paths <span class="token keyword">in</span> /nix/store/4mlqz3gx1n09gbcn95jap6rixpcvzdjd-env<br />checking <span class="token keyword">for</span> references to /build/ <span class="token keyword">in</span> /nix/store/4mlqz3gx1n09gbcn95jap6rixpcvzdjd-env<span class="token punctuation">..</span>.<br />/nix/store/4mlqz3gx1n09gbcn95jap6rixpcvzdjd-env<br /><br />$ <span class="token function">ls</span><br />default.nix main.rs result shell.nix<br /><br />$ readlink result<br />/nix/store/4mlqz3gx1n09gbcn95jap6rixpcvzdjd-env<br /><br />$ ./result/goodbye<br />Goodbye World<span class="token operator">!</span></code></pre>
<p>Well, now you know how to build a package.</p>
<h2 id="what-else%3F" tabindex="-1">What else? <a class="direct-link" href="https://blog.meain.io/2021/intro-to-nix/#what-else%3F">#</a></h2>
<p>Well, so far you know how to create a dev env for working on your project and how to build your own project using nix. This is the core of nix. The best part about the project that you build was that if you change the src to point to a git commit in a hosted repo instead of loading it from your local machine, then you have a fully declarative package under nix. This idea can be extended all the way down to the kernel and things used to build the kernel.</p>
<p>I have been playing around with NixOS and trying to move to it soon hopefully. I also have a personal server which I maintain fully using nix. I have declaratively mentioned the packages I want in there, building any at all if necessary. I can have my nginx config, systemd processes, and all the stuff that I need in that server declaratively mentioned. This is then version controlled using git. Now I have a version controlled state/history of my entire system.</p>
<hr />
<p>This is pretty much the basics of how nix works in the context of building it for working on/building your projects. There is a lot more to nix and I am just scratching the surface here. But I hope that was a good introduction to nix.</p>
<p>Other resources:</p>
<ul>
<li><a href="https://nixos.org/">NixOS</a> - NixOS website</li>
<li><a href="https://nixos.org/guides/how-nix-works.html">How nix works</a> - Quick short intro to working of nix</li>
<li><a href="https://nixos.org/guides/nix-pills/">Nix Pills</a> - A guided introduction to Nix</li>
<li><a href="https://nix.dev/">nix.dev</a> - Guide to using nix</li>
<li><a href="https://www.youtube.com/playlist?list=PLRGI9KQ3_HP_OFRG6R-p4iFgMSK1t5BHs">Nixology | Youtube </a> - A series of videos introducing Nix in a practical way</li>
<li><a href="https://www.youtube.com/watch?v=QKoQ1gKJY5A&list=PL-saUBvIJzOkjAw_vOac75v-x6EzNzZq-">NixOS by WillT | Youtube</a> - A playlist on nixos</li>
<li><a href="https://nixos.org/manual/nix/stable/">Nix Package Manager Guide</a> - A comprehensive guide of the Nix Package Manager</li>
<li><a href="https://nixcloud.io/tour">A tour of Nix</a> - Learn the Nix language itself</li>
<li><a href="https://www.tweag.io/blog/2020-05-25-flakes/">Nix flakes</a> - Nix flakes</li>
<li><a href="https://www.youtube.com/user/elitespartan117j27">John Ringer | Youtube</a> - Great videos on packaging for nix</li>
<li><a href="https://www.youtube.com/channel/UCjqkNrQ8F3OhKSCfCgagWLg">NixCon | Youtube </a> - Nix conference videos</li>
</ul>
<p>Related projects:</p>
<ul>
<li><a href="https://guix.gnu.org/">Guix</a> - GNU fork of nix using Guile scheme as the language</li>
<li><a href="https://direnv.net/">direnv</a> - Automatic shell env when you cd into a project</li>
<li><a href="https://github.com/nix-community/lorri">lorri</a> - Background daemon that builds all your envs</li>
<li><a href="https://github.com/NixOS/nixops">nixops</a> - Manage nixos machines in a cloud env or network</li>
<li><a href="https://www.cachix.org/">Cachix</a> - Caching for nix build artifacts</li>
<li><a href="https://github.com/nmattia/niv/">Niv</a> - Project dependency management</li>
<li><a href="https://github.com/nix-community/home-manager">home-manager</a> - Configure user programs using nix</li>
<li><a href="https://github.com/nix-community/NUR/">NUR</a> - Nix user repository</li>
<li><a href="https://nixery.dev/">Nixery</a> - On the fly docker containers using nix</li>
<li><a href="https://github.com/xtruder/kubenix">Kubenix</a> - Kubernetes resources using nix</li>
</ul>
A useful shell prompt2022-02-12T00:00:00Zhttps://blog.meain.io/2022/my-shell-prompt/<p>I use my shell a lot and have a pretty good prompt which I have built up over the years. It has got to a point where I think I have a pretty good setup and don't find the need to tweak things a lot and so I thought I would blog about it.</p>
<p>For the uninitiated, a shell prompt is the thing that to see to the left(and right in some cases) of place where you type in your commands in the terminal. It will be usually something like <code>project/path $</code>.</p>
<blockquote>
<p>You can find the complete code for my prompt at <a href="https://github.com/meain/dotfiles/blob/master/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme">meain/dotfiles</a></p>
</blockquote>
<p>So yeah, just to begin with here is how my shell prompt looks like when you open a new terminal.</p>
<p><img src="https://blog.meain.io/img/prompt-simple.png" alt="Screenshot of simple prompt" /></p>
<p>Pretty simple, right? I use <a href="https://www.zsh.org/"><code>zsh</code></a> and so I can put stuff on both sides. On the left you have a <code>!</code> mark and on the right you have <code>~</code>. The one on the right, as you might have guessed is showing which directory I am in, in this case <code>~</code> or my <code>$HOME</code> directory. The one of the left, does not mean much as of now, just a marker to tell me where the shell input starts.</p>
<p>Let's make it one level deeper. I am someone who likes to use <code>tmux</code> a lot. I only used to use terminals with <code>tmux</code> until I switched to using Emacs instead of Vim.</p>
<p><img src="https://blog.meain.io/img/prompt-tmux.png" alt="Screenshot of prompt in tmux" /></p>
<p>If I get into a tmux session, the prompt changes from <code>!</code> to <code>=</code>. Just a subtle hint that I am now in a tmux session. On a similar note if I am inside a nix shell, it will switch to using <code>#</code> instead. You can find this piece <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L92-L100">here</a>.</p>
<p>This char also get a red color if previous command fails. I use Vim mode in the shell and so it changes to blue if we are in normal mode. You can find the code <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L145">here</a>.</p>
<p>OK, now let's go into a project. Let me navigate to my blog project.</p>
<p><img src="https://blog.meain.io/img/prompt-project.png" alt="Screenshot showing directory and branch" /></p>
<p>A few things change here. New on the left, along with <code>!</code>, you also have the branch name, in this case <code>master</code>. If you notice, the color of <code>master</code> is blue. This is because I have untracked file. If I add these files to the staging area, then it will turn yellow and once I commit them, it will change to green. I prefer to use colors instead of extra icons to display data if it is possible.</p>
<p>On the right, you will see that we have updated the path. This is always limited to a max of 2 folders. If you look closely, the <code>blog</code> part is in a separate color. This is because <code>blog</code> is the name of the repo on Github.</p>
<p>The Github repo is shown in the path if that string is available there, if it is not available there, then we actually separate it out and show it on the left as a separate block. I did this as in most cases I will be in the root directory and the repo name will be same as the path and didn't want it to take up extra space. You can find the directory/repo display code <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L106-L135">here</a>.</p>
<p><img src="https://blog.meain.io/img/prompt-project-subdir.png" alt="Screenshot showing separate repo block in subdirectory" /></p>
<p>In the above screenshot, since we are inside a nested path which does not contain the repo name, we have separated it out.</p>
<p>Let's look into post commit setup.</p>
<p><img src="https://blog.meain.io/img/prompt-unpushed.png" alt="Screenshot showing lighting on unpushed commits" /></p>
<p>If you notice, we now have a lightning on the right. That is to indicate that we have unpushed commits. You can find that module <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L52-L58">here</a>.</p>
<p>Now to some extra stuff that the prompt does.</p>
<p><img src="https://blog.meain.io/img/prompt-virtualenv.png" alt="Screenshot showing virtualenv marker" /></p>
<p>I used to work a lot with python and so a lot of virtualenv switching. Usually if you were to just enable a virtualenv, it just prepends your prompt with <code>(venv-name)</code>. It was really sucky and so I disabled that and instead, my prompt now show me <code>py:venv-name</code> on the right if I am in a python virtualenv. This is found <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L102-L104">here</a>.</p>
<p><img src="https://blog.meain.io/img/prompt-kubernetes.png" alt="Screenshot showing kubernetes block" /></p>
<p>Also, I work with Kubernetes a lot and one info that I would like to have easy access to is the kubernetes namespace that I am currently in. So, if my shell is a directory with any of the kube files or directories like <code>charts</code>, <code>helm</code> or <code>deployemnt.yaml</code> it will add the info about the namespace in blue on the right. You can find the code <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L71-L75">here</a>.</p>
<p>I have set the prompt to not show me the hostname by default if it is on a local machine, but if not it will show me the hostname as well in the right. Code available <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L77-L79">here</a>.</p>
<p>There are a few modules in there which I don't actively use as of now, but you might find them useful. You can use <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L25-L50">this</a> to display the time since last commit. It might give you an idea if something was recently pulled or not. I used to use <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L81-L83">this</a> to show the commits that I have not yet pushed along with the lighting thing, but decided that was unnecessary information.</p>
<p>I have also added in something to the <code>precmd</code> hook which will let me notify if the previous command took quite some time. You can find the code <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L189-L199">here</a>. The idea here is that if a command is gonna really take a long time to run, I might not be on that terminal when it completes, but I would like to know that it was completed. I could maybe add a voice message or send a IM as well and that way I can even walk away from my laptop and still know that the command completed execution along with the exact command that completed.</p>
<p>Just missed one more tiny hack. I add a special kind of whitespace character to the left prompt <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L141">here</a>. This is combined with a <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/tmux/.tmux.conf#L181">tmux keybinding</a> to search just the prompt and navigate between commands easily instead of aimlessly scrolling and looking for the previous one.</p>
<p>With all of this crap being computed every time a new prompt has to be displayed, you might think that it would be slow. And you are partially correct. What I do is I will display the <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L137-L138">bare minimum</a> when I create a new prompt, <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L155-L176">compute these in separate threads in the background</a> and then once we have it, <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L204-L214">update the prompts</a>. This seems to work really well for me. If you see, I don't directly assign the <code>PS1</code> and <code>RPS1</code> directly with the full blown stuff, but instead have the very basic. I have two function <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L140-L146"><code>generate_rprompt</code></a> and <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L148-L150"><code>generate_lprompt</code></a> both of which run separately in the background computing the right and left prompt respectively. Now when the <code>generate_lprompt</code> finishes computation, the result is written to a temp file and <a href="https://github.com/meain/dotfiles/blob/5934c3c42aa8e4ed545ce380548bfaef20b8a2f9/zsh/.config/zsh/.zsh-custom/themes/snipe.zsh-theme#L158">sends a <code>USR1</code> signal</a> which we pick up in the parent process and update the actual left prompt from the temp file. Similar thing is done for the right prompt as well. This way, we don't really block the user from being able to type while we compute stuff.</p>
<p>And there you go. That is most of the stuff in my prompt. It is kinda a lot of random information dumped here, but hopefully this shows you the kind of stuff that you can do in your shell.</p>
<p>For those who don't wanna spend so much time customizing stuff, you can use something prebuilt. You might not be able to get exactly what you want, but most of them are pretty powerful. Let me list down a few ones that might be good:</p>
<ul>
<li><a href="https://github.com/romkatv/powerlevel10k">Powerlevel10k</a> Successor of the classic Powerlevel9k</li>
<li><a href="https://github.com/ohmyzsh/ohmyzsh/wiki/Themes">oh-my-zsh themes</a> I have used some of them to see how they implement things</li>
<li><a href="https://starship.rs/">starship</a> I hear this is very popular</li>
<li><a href="https://github.com/geometry-zsh/geometry">geometry</a> Looks nice</li>
</ul>
Drag and drop from terminal2022-02-23T00:00:00Zhttps://blog.meain.io/2022/terminal-drag-and-drop/<p>I have built up a pretty good workflow in the terminal. The main thing that I was missing until now was an easy way to drag and drop the files that I am looking at in a terminal to other apps.</p>
<p>So far, whenever I wanted to share a file from the terminal I would open up a GUI file browser, navigate to that directory, find the file and then drag and drop it. Not anymore.
I recently was able to cobble together a pretty good(IMO) for dragging and dropping files to GUI applications and thought I would share.</p>
<p><video src="https://meain.io/blog-videos/videos/dnd.mp4" controls="">Drag and drop workflow</video></p>
<p>Here is what is happening in the video. I am in a project repo in which I have listed the files in the current directory using <code>ls</code>.
Now I just double-click on the file that I want to share, it bring up a thing (<code>dragon</code>) from which I can just drag and drop into slack.</p>
<p>Now let us see how to get this workflow.
The main tool that is helping with this is <a href="https://github.com/mwh/dragon">dragon</a>. Here is how you use it:</p>
<pre class="language-shell"><code class="language-shell">dragon <span class="token operator"><</span>filename<span class="token operator">></span></code></pre>
<p>Using the above will open a window from which you can drag and drop. You can pass <code>-x</code> to close it automatically after you share one item.</p>
<p>It is already great on its own, but let's wire it up so that we can bring this up when we double-click on a filename.
This is done using tmux.</p>
<pre><code>bind -n DoubleClick1Pane run-shell "dragon -x '#{pane_current_path}/#{mouse_word}'"
</code></pre>
<p>What is happening here is that when you double-click on something, tmux end up calling dragon with <code>pane_current_path</code>(pwd of current shell) + <code>/</code> + <code>mouse_word</code>(whitespace separated piece of text under cursor) which ends up as path to the file.
<em>You can probably have it be more intelligent by separately passing it onto a script that only adds <code>pane_current_path</code> if and only if <code>mouse_word</code> is not a file or even search backwards/forwards for that file, but I just went with a simpler implementation as I will be mostly selecting after typing ls.</em></p>
<p>Another possibility is that you can just wire it up with <a href="https://github.com/ranger/ranger">ranger</a> or <a href="https://github.com/gokcehan/lf">lf</a> and open dragon up like that.</p>
<p>I do end up taking and sharing a lot of screenshots, and so I have a special case for that.
Every time I take a screenshot, the path is automatically copied to my clipboard. Now, in my wm (<a href="https://github.com/i3/i3">i3</a>), I have a keybinding that will open up the path in clipboard with dragon. Here is the code for that:</p>
<pre><code>bindsym $mod+Mod1+Control+Shift+l exec "dragon -x $(pbpaste)"
</code></pre>
<p>With this, after I take a screenshot I can press <code>$mod+Mod1+Control+Shift+l</code> and have that window to share it.
<em>About the keybinding, it actually ends up as tab+shift+l for me after remappings for those wondering how on earth I am remembering/pressing them</em></p>
<p><strong>BONUS</strong>: The above explained setup is Linux only, but if you are on macOS, I was using something really hacky with hammerspoon to do something similar: <a href="https://github.com/meain/dotfiles/blob/master/hammerspoon/.config/hammerspoon/slackdrop.lua">https://github.com/meain/dotfiles/blob/master/hammerspoon/.config/hammerspoon/slackdrop.lua</a> .
This essentially just end up emulating the keybindings necessary to upload a file (from path in clipboard) from the slack UI.</p>
<p>And that is it. Enjoy sticking to just a terminal.</p>
Your terminal on lsd2022-03-26T00:00:00Zhttps://blog.meain.io/2022/your-terminal-on-lsd/<p>First of all, sorry about the clickbait title, but I could not stop myself from using this overused joke. What I wanted to talk about in this article is about <a href="https://github.com/Peltoche/lsd">Peltoche/lsd</a>. <code>lsd</code> is a rewrite of the classic <code>ls</code> command but with a few new and useful(IMO) features. I am the primary maintainer of this project as of now and thought I would go over some interesting features.</p>
<p>I have seen a few videos, blogs and podcasts that talk about <code>lsd</code>, but almost all of them just tend to talk about colors and icons. While they are awesome, there is a lot more awesome features in lsd that you should check out.</p>
<p>Before we begin, let me show you how lsd looks for me.</p>
<p><img src="https://blog.meain.io/img/personal-lsd.png" alt="Screenshot of lsd" /></p>
<p>If you use lsd or if you have seen the screenshot in the README, you can tell it is quite different. Btw, this article is nothing that a quick read of the <code>--help</code> can't reaplace, but I thought I would highlight the things that I like. This is just gonna be a list of thing I like in no particular order.</p>
<h1 id="--blocks" tabindex="-1"><code>--blocks</code> <a class="direct-link" href="https://blog.meain.io/2022/your-terminal-on-lsd/#--blocks">#</a></h1>
<p>In most cases when you use <code>ls -l</code> command, you don't really need all the information it gives you. At least for me, what I need is usually just the date, size, permissions and the name of the file. <code>lsd</code> lets you pick and choose just what you want.</p>
<p>You can pick and choose any of these in any order:</p>
<ul>
<li>permission: permissions for the file</li>
<li>user: user name</li>
<li>group: group name</li>
<li>size: size</li>
<li>date: date last modified</li>
<li>name: file name</li>
<li>inode: inode number</li>
<li>links: number of hard links</li>
</ul>
<p>For example, I can do something like <code>lsd --block date,name,size</code> and get something like this:</p>
<p><img src="https://blog.meain.io/img/lsd-blocks.png" alt="Custom blocks with lsd" /></p>
<p>Or let's say I don't care about the name, I just want the sizes ordered by date. I could use something like:</p>
<p><img src="https://blog.meain.io/img/lsd-custom-blocks.png" alt="Selected blocks with sort" /></p>
<p><em>I am not necessarily saying this is useful, just that it is possible.</em></p>
<h1 id="--tree" tabindex="-1"><code>--tree</code> <a class="direct-link" href="https://blog.meain.io/2022/your-terminal-on-lsd/#--tree">#</a></h1>
<p>Another nice thing is its ability to do trees. This by itself is not that useful, but you can combine it with custom <code>blocks</code> which becomes really powerful.</p>
<p><img src="https://blog.meain.io/img/lsd-tree-long.png" alt="Tree with long" /></p>
<p>Also, you can always sort this by time or size or anything.</p>
<h1 id="--date-relative" tabindex="-1"><code>--date relative</code> <a class="direct-link" href="https://blog.meain.io/2022/your-terminal-on-lsd/#--date-relative">#</a></h1>
<p>This is one of my favourite features. I find it easier to understand relative time than have an exact date in most cases as I can be sometimes ignorant of what day today is. With <code>--date relative</code> I can continue to be ignorant and not fix the actual problem. <em>It is possible that I am partial to this because this was one of the first features that I added in</em>.</p>
<p><img src="https://blog.meain.io/img/lsd-date-relative.png" alt="Relative date" /></p>
<h1 id="--permission-octal" tabindex="-1"><code>--permission octal</code> <a class="direct-link" href="https://blog.meain.io/2022/your-terminal-on-lsd/#--permission-octal">#</a></h1>
<p>This is one of the newest features and might not be in a release by the time you read this, but you should be able to get it if you build from master. This lets you view the permissions in octal instead of as read write execute thingies. Here is what it looks like as of now.</p>
<p><img src="https://blog.meain.io/img/lsd-octal-perm.png" alt="Ocatl permissions" /></p>
<p><em>If you see the prompt, you will see that I am in the <code>octal-perm</code> branch wile I was working on this post.</em></p>
<h1 id="size%2Fdate-based-colors" tabindex="-1">Size/date based colors <a class="direct-link" href="https://blog.meain.io/2022/your-terminal-on-lsd/#size%2Fdate-based-colors">#</a></h1>
<p>If you haven't noticed already, both the size column and the date column changes colors based on what it contains. Size column changes color based on how big/small a file is and date column changes color based on when a file was last modified. The best part is that you can change all the colors used here via the config file.</p>
<p><img src="https://blog.meain.io/img/lsd-diff-size.png" alt="Different size lsd" /></p>
<h1 id="themeing-and-config-file" tabindex="-1">Themeing and config file <a class="direct-link" href="https://blog.meain.io/2022/your-terminal-on-lsd/#themeing-and-config-file">#</a></h1>
<p>Building on the last one, we support having a config file for all your customizations. Any flags that you wanna set by default, you can add into the config file. You can always ignore the config file with <code>--ignore-config</code> cli flag and start from scratch. You might have already seen me using that flag for the screenshots (I was too lazy to move my config file).</p>
<p>You can also theme lsd completely using the theme file. You can change the color of any block, and the differing levels of sizes or dates. I have a really muted theme as you might have seen from the initial screenshot. You can take a look at my config <a href="https://github.com/meain/dotfiles/tree/master/lsd/.config/lsd">here</a>.</p>
<p>Refer to the README to know what all options are available.</p>
<h1 id="support-for-ls_colors" tabindex="-1">Support for <code>LS_COLORS</code> <a class="direct-link" href="https://blog.meain.io/2022/your-terminal-on-lsd/#support-for-ls_colors">#</a></h1>
<p>We support <code>LS_COLORS</code>. For those of you who don't know what it is, you can think of it as a common convention on how to color filenames. Using <code>LS_COLORS</code> you can define what color a directory should be, what color symlinks should be, what color <code>*.md</code> files should be, etc. Even better is that there is already people who have gone ahead and created different colorschemes. There are also tools that help you create custom color schemes. For example if you want absolutely everything color coded, you can use <a href="https://github.com/trapd00r/LS_COLORS">trapd00r/LS_COLORS</a>. It will look something like this with all the icons.</p>
<p><img src="https://blog.meain.io/img/lsd-bling.png" alt="All colors and icons" /></p>
<p>Checkout these projects if you are interested:</p>
<ul>
<li><a href="https://github.com/trapd00r/LS_COLORS">https://github.com/trapd00r/LS_COLORS</a></li>
<li><a href="https://geoff.greer.fm/lscolors/">https://geoff.greer.fm/lscolors/</a></li>
<li><a href="https://github.com/sharkdp/vivid">https://github.com/sharkdp/vivid</a></li>
</ul>
<h1 id="icons" tabindex="-1">Icons <a class="direct-link" href="https://blog.meain.io/2022/your-terminal-on-lsd/#icons">#</a></h1>
<p>Oh, almost forgot. We got icons, but you knew that already.</p>
<p>And I think I am gonna wrap up with that. Thanks to all the contributors of lsd and Peltoche for starting the project :D.</p>
Playing around with tree-sitter in Emacs2022-05-18T00:00:00Zhttps://blog.meain.io/2022/more-treesitter-emacs/<p>Tree sitter has been one of those tools that has enabled me to add in a lot of useful features to my dev environment. I have previously <a href="https://blog.meain.io/2021/releasing-evil-textobj-tree-sitter/">written</a> about my Emacs package <a href="https://github.com/meain/evil-textobj-tree-sitter/">meain/evil-textobj-tree-sitter</a> which will let you operate on language constructs like functions, classes, loops etc as evil <code>textobjects</code>. I have been playing around with tree-sitter a lot more since and in this blog I wanted to share some interesting thing that I have been using tree-sitter for.</p>
<h1 id="navigation-between-textobjects" tabindex="-1">Navigation between textobjects <a class="direct-link" href="https://blog.meain.io/2022/more-treesitter-emacs/#navigation-between-textobjects">#</a></h1>
<p><video src="https://meain.io/blog-videos/videos/tree-sitter-navigate.mp4" controls="">Navigate between tree-sitter objects</video></p>
<p>Before we go into my custom configs, here is a new feature that got added into the package. As of now you can navigate to the next textobject of a different types. For example, you can navigate to the next function or loop. <a href="https://github.com/meain/dotfiles/blob/34ef5e3331757ac32dd066f5baa54f76cf78211b/emacs/.config/emacs/init.el#L2237-L2256">Here</a> is my Emacs config on top of the package functionality to go to the next textobject. I have found this to be really useful to go through all the functions in the file. It comes in really handy in case of test files where go to next function is kind a like go to next test case. That combined with "run current test" is really powerful. Just in case anyone is interested, <a href="https://github.com/meain/toffee/">meain/toffee</a> is what I am using to run tests under the cursor combined with a <a href="https://github.com/meain/dotfiles/blob/master/emacs/.config/emacs/init.el#L1591-L1614">tiny bit</a> of Emacs config.</p>
<h1 id="show-current-class%2Ffunction-name-in-modeline" tabindex="-1">Show current class/function name in modeline <a class="direct-link" href="https://blog.meain.io/2022/more-treesitter-emacs/#show-current-class%2Ffunction-name-in-modeline">#</a></h1>
<p>We use tree-sitter to parse the current class/fucntion that you are in and display it in the modeline. I have been thinking about maybe adding it to the above mentioned package or maybe even create a separate <code>tree-sitter-tools</code> package, but have never got around to it. <a href="https://github.com/meain/dotfiles/blob/34ef5e3331757ac32dd066f5baa54f76cf78211b/emacs/.config/emacs/init.el#L2167-L2186">Here</a> is the function that does the heavy lifting and <a href="https://github.com/meain/dotfiles/blob/34ef5e3331757ac32dd066f5baa54f76cf78211b/emacs/.config/emacs/init.el#L3103-L3116">here</a> is the code that adds it to the modeline(or header line in my case).</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token keyword">setq</span> meain/tree-sitter-calss-like <span class="token punctuation">'(</span><span class="token punctuation">(</span><span class="token car">rust-mode</span> <span class="token punctuation">.</span> <span class="token punctuation">(</span><span class="token car">impl_item</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">python-mode</span> <span class="token punctuation">.</span> <span class="token punctuation">(</span><span class="token car">class_definition</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token keyword">setq</span> meain/tree-sitter-function-like <span class="token punctuation">'(</span><span class="token punctuation">(</span><span class="token car">rust-mode</span> <span class="token punctuation">.</span> <span class="token punctuation">(</span><span class="token car">function_item</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">go-mode</span> <span class="token punctuation">.</span> <span class="token punctuation">(</span><span class="token car">function_declaration</span> method_declaration<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">python-mode</span> <span class="token punctuation">.</span> <span class="token punctuation">(</span><span class="token car">function_definition</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/tree-sitter-thing-name</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">kind</span></span><span class="token punctuation">)</span></span><br /><span class="token string">"Get name of tree-sitter <span class="token argument">THING-KIND</span>."</span><br /><span class="token punctuation">(</span><span class="token keyword">if</span> tree-sitter-mode<br /> <span class="token punctuation">(</span><span class="token keyword">let*</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">node-types-list</span> <span class="token punctuation">(</span><span class="token car">pcase</span> kind<br /> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'class-like</span> meain/tree-sitter-calss-like<span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'function-like</span> meain/tree-sitter-function-like<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">node-types</span> <span class="token punctuation">(</span><span class="token car">alist-get</span> major-mode node-types-list<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> node-types<br /> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">node-at-point</span> <span class="token punctuation">(</span><span class="token car">car</span> <span class="token punctuation">(</span><span class="token car">remove-if</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token car">eq</span> <span class="token boolean">nil</span> x<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">seq-map</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token car">tree-sitter-node-at-point</span> x<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> node-types<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> node-at-point<br /> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">node-name-node-at-point</span> <span class="token punctuation">(</span><span class="token car">tsc-get-child-by-field</span> node-at-point '<span class="token lisp-property property">:name</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> node-name-node-at-point<br /> <span class="token punctuation">(</span><span class="token car">tsc-node-text</span> node-name-node-at-point<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p>It only supports go, rust and python as of now but should be easy enuogh to add more languages. Just to give you an idea of what the code does, it looks for the node of a specific the type defined by <code>meain/tree-sitter-calss-like</code> or <code>meain/tree-sitter-function-like</code> at the current cursor position. From this node, we extract the field named <code>name</code> which would give us the name of the class or function. Just a heads up for people planning to use it in the modeline. Although tree-sitter is pretty fast, you might want to avoid updating this continuously in the modeline. That is why I make use of <a href="https://codeberg.org/ideasman42/emacs-mode-line-idle">mode-line-idle</a> so that the computation is deferred to when the cursor is idle.</p>
<h1 id="narrow-to-language-level-constructs" tabindex="-1">Narrow to language level constructs <a class="direct-link" href="https://blog.meain.io/2022/more-treesitter-emacs/#narrow-to-language-level-constructs">#</a></h1>
<p><video src="https://meain.io/blog-videos/videos/tree-sitter-narrow.mp4" controls="">Narrowign to tree-sitter objects</video></p>
<p>I don't use this a lot, but I am glad when I have to use it that I have it. It is useful when you are concentrating on a single function or loop. It also comes in handy when showing/explaining code to someone over a call. I would usually be talking in a function context and being able to easily narrow to that function is really useful.</p>
<p>Just FYI, I don't use the builtin narrow, but use <a href="https://github.com/Malabarba/fancy-narrow">fancy-narrow</a> instead. The package is not actively maintained anymore, but it works pretty well. For this, we make use of a util function from <a href="https://github.com/meain/evil-textobj-tree-sitter/">meain/evil-textobj-tree-sitter</a> package to get the range of the textobject, and is passed over to fancy-narrow to narrow to it. You can find the code for it below.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token comment">;; Fancy narrow to textobj</span><br /><span class="token punctuation">(</span><span class="token keyword">use-package</span> emacs<br /> <span class="token lisp-property property">:commands</span> <span class="token punctuation">(</span><span class="token car">meain/fancy-narrow-to-thing</span><span class="token punctuation">)</span><br /> <span class="token lisp-property property">:config</span><br /> <span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/fancy-narrow-to-thing</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">thing</span></span><span class="token punctuation">)</span></span><br /> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token car">buffer-narrowed-p</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">fancy-widen</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">range</span> <span class="token punctuation">(</span><span class="token car">evil-textobj-tree-sitter--range</span> <span class="token number">1</span> <span class="token punctuation">(</span><span class="token car">list</span> <span class="token punctuation">(</span><span class="token car">intern</span> thing<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">fancy-narrow-to-region</span> <span class="token punctuation">(</span><span class="token car">car</span> range<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">cdr</span> range<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">evil-leader/set-key</span> <span class="token string">"n n"</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">fancy-widen</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">evil-leader/set-key</span> <span class="token string">"n f"</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">meain/fancy-narrow-to-thing</span> <span class="token string">"function.outer"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">evil-leader/set-key</span> <span class="token string">"n c"</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">meain/fancy-narrow-to-thing</span> <span class="token string">"class.outer"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">evil-leader/set-key</span> <span class="token string">"n C"</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">meain/fancy-narrow-to-thing</span> <span class="token string">"comment.outer"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">evil-leader/set-key</span> <span class="token string">"n o"</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">meain/fancy-narrow-to-thing</span> <span class="token string">"loop.outer"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">evil-leader/set-key</span> <span class="token string">"n i"</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">meain/fancy-narrow-to-thing</span> <span class="token string">"conditional.outer"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">evil-leader/set-key</span> <span class="token string">"n a"</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">meain/fancy-narrow-to-thing</span> <span class="token string">"parameter.outer"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p><em>And that's it for now.</em></p>
Navigating config files using tree-sitter2022-07-12T00:00:00Zhttps://blog.meain.io/2022/navigating-config-files-using-tree-sitter/<p>Yet another entry for <code>tree-sitter</code>. For those new here, I have blogged previously about using <code>tree-sitter</code> <a href="https://blog.meain.io/2021/releasing-evil-textobj-tree-sitter/">here</a> and <a href="https://blog.meain.io/2022/more-treesitter-emacs/">here</a>.</p>
<p>Here is another installment of how I use <code>tree-sitter</code> to simplify/speed up things for me. In this installment we are going to see how we can use <code>tree-sitter</code> to easily navigate around config files.</p>
<p>When having to navigate through huge config files there are two problems that I usually run into.</p>
<ul>
<li>Not being able to tell where I am currently at (what are the parents)</li>
<li>Being able to go to a specific nested entry</li>
</ul>
<p>Now let's see how we can fix these. In the below image you can see the path is shown in the <code>header-line</code> and that we are using <a href="https://www.emacswiki.org/emacs/ImenuMode">imenu</a> and <a href="https://github.com/minad/consult">consult-imenu</a> to go to a node with completion.</p>
<p><img src="https://blog.meain.io/img/tree-sitter-nav.png" alt="Screenshot of it working" /></p>
<h1 id="figuring-out-where-you-are" tabindex="-1">Figuring out where you are <a class="direct-link" href="https://blog.meain.io/2022/navigating-config-files-using-tree-sitter/#figuring-out-where-you-are">#</a></h1>
<p>As you might know, <code>tree-sitter</code> has super sweet syntax tree which we can use. We can just query that to get the info on the file structure.</p>
<p>Just to give you a high level idea of what the code is doing, we query the entire tree to figure out where all the keys are. For example in case of <code>json</code>, you can use a query like <code>(object (pair (string (string_content) @key) (_)) @item)</code> to get the keys and the objects that they represent. Once we have that, we just go through them and find out all the <code>item</code> thingies for which start is before us and end is after us and get their keys.</p>
<p>The below snippet is pretty much what you can use for this. You can drop this into the statusline probably with something that will let you do deferred computation like <a href="https://codeberg.org/ideasman42/emacs-mode-line-idle">mode-line-idle</a></p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/tree-sitter-config-nesting</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">or</span> <span class="token punctuation">(</span><span class="token car">eq</span> major-mode <span class="token quoted-symbol variable symbol">'json-mode</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">eq</span> major-mode <span class="token quoted-symbol variable symbol">'yaml-mode</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">eq</span> major-mode <span class="token quoted-symbol variable symbol">'nix-mode</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">let*</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">cur-point</span> <span class="token punctuation">(</span><span class="token car">point</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">query</span> <span class="token punctuation">(</span><span class="token car">pcase</span> major-mode<br /> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'json-mode</span> <span class="token string">"(object (pair (string (string_content) @key) (_)) @item)"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'yaml-mode</span> <span class="token string">"(block_mapping_pair (flow_node) @key (_)) @item"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'nix-mode</span> <span class="token string">"(bind (attrpath (attr_identifier) @key)) @item"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">root-node</span> <span class="token punctuation">(</span><span class="token car">tsc-root-node</span> tree-sitter-tree<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">query</span> <span class="token punctuation">(</span><span class="token car">tsc-make-query</span> tree-sitter-language query<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">matches</span> <span class="token punctuation">(</span><span class="token car">tsc-query-matches</span> query root-node <span class="token quoted-symbol variable symbol">#'tsc--buffer-substring-no-properties</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">string-join</span> <span class="token punctuation">(</span><span class="token car">remove-if</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span> <span class="token punctuation">(</span><span class="token car">eq</span> x <span class="token boolean">nil</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">seq-map</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span><br /> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">(</span><br /> <span class="token punctuation">(</span><span class="token car">item</span> <span class="token punctuation">(</span><span class="token car">seq-elt</span> <span class="token punctuation">(</span><span class="token car">cdr</span> x<span class="token punctuation">)</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">key</span> <span class="token punctuation">(</span><span class="token car">seq-elt</span> <span class="token punctuation">(</span><span class="token car">cdr</span> x<span class="token punctuation">)</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">and</span><br /> <span class="token punctuation">(</span><span class="token car">></span> cur-point <span class="token punctuation">(</span><span class="token car">byte-to-position</span> <span class="token punctuation">(</span><span class="token car">car</span> <span class="token punctuation">(</span><span class="token car">tsc-node-byte-range</span> <span class="token punctuation">(</span><span class="token car">cdr</span> item<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car"><</span> cur-point <span class="token punctuation">(</span><span class="token car">byte-to-position</span> <span class="token punctuation">(</span><span class="token car">cdr</span> <span class="token punctuation">(</span><span class="token car">tsc-node-byte-range</span> <span class="token punctuation">(</span><span class="token car">cdr</span> item<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">format</span> <span class="token string">"%s"</span> <span class="token punctuation">(</span><span class="token car">tsc-node-text</span> <span class="token punctuation">(</span><span class="token car">cdr</span> key<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token boolean">nil</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> matches<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token string">"."</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p><em>Btw, if you are lost in life, <code>tree-sitter</code> can't help you yet unfortunately.</em></p>
<h1 id="going-to-a-node" tabindex="-1">Going to a node <a class="direct-link" href="https://blog.meain.io/2022/navigating-config-files-using-tree-sitter/#going-to-a-node">#</a></h1>
<p>For this, I initially had a <a href="https://github.com/meain/dotfiles/blob/45e01a18b7d6f2f516059a1a0cfa162099aa674f/emacs/.config/emacs/init.el#L2270-L2312">function leveraging <code>completing-read</code></a>, but since then I ended up just implementing logic to populate <a href="https://www.emacswiki.org/emacs/ImenuMode">imenu</a> list and let imenu do it along with the help of <a href="https://github.com/minad/consult">conult-imenu</a>.</p>
<p>Here is the code which does it:</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/get-config-nesting-paths</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span><br /><span class="token string">"Get out all the nested paths in a config file."</span><br /><span class="token punctuation">(</span><span class="token keyword">let*</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">query</span> <span class="token punctuation">(</span><span class="token car">pcase</span> major-mode<br /> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'json-mode</span> <span class="token string">"(object (pair (string (string_content) @key) (_)) @item)"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'yaml-mode</span> <span class="token string">"(block_mapping_pair (flow_node) @key (_)) @item"</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token quoted-symbol variable symbol">'nix-mode</span> <span class="token string">"(bind (attrpath (attr_identifier) @key)) @item"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">root-node</span> <span class="token punctuation">(</span><span class="token car">tsc-root-node</span> tree-sitter-tree<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">query</span> <span class="token punctuation">(</span><span class="token car">tsc-make-query</span> tree-sitter-language query<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">matches</span> <span class="token punctuation">(</span><span class="token car">tsc-query-matches</span> query root-node <span class="token quoted-symbol variable symbol">#'tsc--buffer-substring-no-properties</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">prev-node-ends</span> <span class="token punctuation">'(</span><span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment">;; we can get away with just end as the list is sorted</span><br /> <span class="token punctuation">(</span><span class="token car">current-key-depth</span> <span class="token punctuation">'(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">item-ranges</span> <span class="token punctuation">(</span><span class="token car">seq-map</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span><br /> <span class="token punctuation">(</span><span class="token keyword">let</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">item</span> <span class="token punctuation">(</span><span class="token car">seq-elt</span> <span class="token punctuation">(</span><span class="token car">cdr</span> x<span class="token punctuation">)</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">key</span> <span class="token punctuation">(</span><span class="token car">seq-elt</span> <span class="token punctuation">(</span><span class="token car">cdr</span> x<span class="token punctuation">)</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">list</span> <span class="token punctuation">(</span><span class="token car">tsc-node-text</span> <span class="token punctuation">(</span><span class="token car">cdr</span> key<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">tsc-node-range</span> <span class="token punctuation">(</span><span class="token car">cdr</span> key<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">tsc-node-range</span> <span class="token punctuation">(</span><span class="token car">cdr</span> item<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> matches<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span><br /> <span class="token punctuation">(</span><span class="token keyword">let*</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">current-end</span> <span class="token punctuation">(</span><span class="token car">seq-elt</span> <span class="token punctuation">(</span><span class="token car">cadr</span> <span class="token punctuation">(</span><span class="token car">cdr</span> x<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">parent-end</span> <span class="token punctuation">(</span><span class="token car">car</span> prev-node-ends<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">current-key</span> <span class="token punctuation">(</span><span class="token car">car</span> x<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">progn</span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token car">></span> current-end parent-end<span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span><br /> <span class="token punctuation">(</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token car">></span> current-end x<span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">progn</span><br /> <span class="token punctuation">(</span><span class="token keyword">setq</span> prev-node-ends <span class="token punctuation">(</span><span class="token car">cdr</span> prev-node-ends<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">setq</span> current-key-depth <span class="token punctuation">(</span><span class="token car">cdr</span> current-key-depth<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> prev-node-ends<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">setq</span> current-key-depth <span class="token punctuation">(</span><span class="token keyword">cons</span> current-key current-key-depth<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token keyword">setq</span> prev-node-ends <span class="token punctuation">(</span><span class="token keyword">cons</span> current-end prev-node-ends<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">list</span> <span class="token punctuation">(</span><span class="token car">reverse</span> current-key-depth<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">seq-elt</span> <span class="token punctuation">(</span><span class="token car">cadr</span> x<span class="token punctuation">)</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> item-ranges<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/goto-config-nesting-path</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span><br /><span class="token string">"Interactively go to a nested path in a config file."</span><br /><span class="token punctuation">(</span><span class="token interactive keyword">interactive</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token keyword">let*</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token car">paths</span> <span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span><br /> <span class="token punctuation">(</span><span class="token keyword">cons</span> <span class="token punctuation">(</span><span class="token car">string-join</span> <span class="token punctuation">(</span><span class="token car">car</span> x<span class="token punctuation">)</span> <span class="token string">"."</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">cadr</span> x<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">meain/get-config-nesting-paths</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">goto-char</span> <span class="token punctuation">(</span><span class="token car">cdr</span> <span class="token punctuation">(</span><span class="token car">assoc</span><br /> <span class="token punctuation">(</span><span class="token car">completing-read</span> <span class="token string">"Choose path: "</span> paths<span class="token punctuation">)</span><br /> paths<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /><span class="token punctuation">(</span><span class="token defun"><span class="token keyword">defun</span> <span class="token function">meain/imenu-config-nesting-path</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span><br /><span class="token string">"Return config-nesting paths for use in imenu"</span><br /><span class="token punctuation">(</span><span class="token car">mapcar</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"><span class="token argument variable">x</span></span><span class="token punctuation">)</span></span><br /> <span class="token punctuation">(</span><span class="token keyword">cons</span> <span class="token punctuation">(</span><span class="token car">string-join</span> <span class="token punctuation">(</span><span class="token car">car</span> x<span class="token punctuation">)</span> <span class="token string">"."</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token car">cadr</span> x<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token punctuation">(</span><span class="token car">meain/get-config-nesting-paths</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p>The idea with this is more or less the same. We fetch the entire list of items, then go through it and create a list of items and their locations. Now we feed this to imenu by using something like below and we are good to go.</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token car">add-hook</span> <span class="token quoted-symbol variable symbol">'nix-mode-hook</span> <span class="token punctuation">(</span><span class="token lambda"><span class="token keyword">lambda</span> <span class="token punctuation">(</span><span class="token arguments"></span><span class="token punctuation">)</span></span><br /> <span class="token punctuation">(</span><span class="token keyword">setq</span> imenu-create-index-function <span class="token quoted-symbol variable symbol">#'meain/imenu-config-nesting-path</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre>
<p>And viola, there you go. I wanted to write a longer blog, but after I got this running I got really excited that I spent all the energy on celebrating. Even still, I hope folks find it useful. I am kinda thinking about creating another Emacs package around it, as in navigating and viewing for all kind of files like config / code / documentation (md) etc. But until then, you are welcome to copy paste the code from my <a href="https://github.com/meain/dotfiles">dotfiles</a>.</p>
Self hosted location history2022-09-17T00:00:00Zhttps://blog.meain.io/2022/self-hosted-location-history/<p>I am a big fan of being able to view the history of places that I have been to. <a href="https://www.tomsguide.com/how-to/how-to-view-location-history-in-google-maps">Google location history</a> is great in this regards. It records all the places you and you can view this on Google maps. But for some time now I have been removing Google more and more from my life and one of the first things to go was location history along with search and YouTube history. While I don't miss the others, not being able to view the location history has been a bummer.</p>
<p><img src="https://blog.meain.io/img/google-location-history.png" alt="" /></p>
<p><em>This is not my screenshot btw ;)</em></p>
<p>I was aware of possible alternatives, self hostable options but never got around to doing it. I finally decided to spend some time over this weekend and setup <a href="https://github.com/OwnTracks">OwnTracks</a> on my sever. It was a pretty painless setup. Went way smoother than I expected to be frank. I did a bit of research and after a while the two main options that I could find was OwnTracks and <a href="https://apps.nextcloud.com/apps/phonetrack">PhoneTrack from Nextcloud</a>. While Nextcloud is pretty good, I did not want to setup a Nextcloud instance for this use case and so decided to go with OwnTracks.</p>
<h1 id="why-track-yourself%3F" tabindex="-1">Why track yourself? <a class="direct-link" href="https://blog.meain.io/2022/self-hosted-location-history/#why-track-yourself%3F">#</a></h1>
<ul>
<li>You can find out where your phone is if you misplace it</li>
<li>Kinda interesting to know where all you have been to</li>
<li>Good way to know at what time you were at a certain place</li>
<li>It is fun to hoard data</li>
</ul>
<h1 id="setting-it-up" tabindex="-1">Setting it up <a class="direct-link" href="https://blog.meain.io/2022/self-hosted-location-history/#setting-it-up">#</a></h1>
<p>Now that the initial fluff is done, let me get into the meat of this blog, how to self host OwnTracks. It consists of the following steps:</p>
<ul>
<li>Setup <a href="https://github.com/owntracks/recorder">recorder</a> as OwnTracks server</li>
<li>Configure <a href="https://nginx.org/en/">nginx</a> to forward requests with auth</li>
<li>Install and setup <a href="https://play.google.com/store/apps/details?id=org.owntracks.android&gl=US">OwnTracks Android app</a> (iOS if you are from that island)</li>
<li>Setup and configure OwnTracks <a href="https://github.com/owntracks/frontend">frontend</a> for viewing location history</li>
</ul>
<h2 id="setting-up-recorder" tabindex="-1">Setting up recorder <a class="direct-link" href="https://blog.meain.io/2022/self-hosted-location-history/#setting-up-recorder">#</a></h2>
<p>I personally use <a href="https://nixos.org/">nix</a> to setup my server. I could not find OwnTracks on nixpkgs and so the first step for me is to package OwnTracks is to package it under nix. You can find my derivation on <a href="https://github.com/nix-community/nur-combined/blob/master/repos/meain/pkgs/otrecorder/default.nix">nur</a> under meain.</p>
<pre class="language-nix"><code class="language-nix"><span class="token punctuation">{</span> pkgs<span class="token punctuation">,</span> lib<span class="token punctuation">,</span> fetchFromGitHub<span class="token punctuation">,</span> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span> <span class="token punctuation">}</span><span class="token punctuation">:</span><br /><br />pkgs<span class="token punctuation">.</span>stdenv<span class="token punctuation">.</span>mkDerivation rec <span class="token punctuation">{</span><br /> pname <span class="token operator">=</span> <span class="token string">"otrecorder"</span><span class="token punctuation">;</span><br /> name <span class="token operator">=</span> pname<span class="token punctuation">;</span><br /> version <span class="token operator">=</span> <span class="token string">"0.9.1"</span><span class="token punctuation">;</span><br /><br /> src <span class="token operator">=</span> fetchFromGitHub <span class="token punctuation">{</span><br /> owner <span class="token operator">=</span> <span class="token string">"owntracks"</span><span class="token punctuation">;</span><br /> repo <span class="token operator">=</span> <span class="token string">"recorder"</span><span class="token punctuation">;</span><br /> rev <span class="token operator">=</span> version<span class="token punctuation">;</span><br /> sha256 <span class="token operator">=</span> <span class="token string">"sha256-/y74jfofvWTcHSX+9wtrCRclaS5Aw03TCz11mrZiqiM="</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br /> nativeBuildInputs <span class="token operator">=</span> <span class="token keyword">with</span> pkgs<span class="token punctuation">;</span> <span class="token punctuation">[</span> mosquitto curl lmdb libconfig libuuid <span class="token punctuation">]</span><span class="token punctuation">;</span><br /><br /> configurePhase <span class="token operator">=</span> <span class="token string">''<br /> cp config.mk.in config.mk<br /> ''</span><span class="token punctuation">;</span><br /><br /> installPhase <span class="token operator">=</span> <span class="token string">''<br /> mkdir -p $out/bin $out/usr/share/ot-recorder<br /> cp ot-recorder $out/bin/ot-recorder<br /> cp -R docroot/* $out/usr/share/ot-recorder<br /> ''</span><span class="token punctuation">;</span><br /><br /> meta <span class="token operator">=</span> <span class="token keyword">with</span> lib<span class="token punctuation">;</span> <span class="token punctuation">{</span><br /> description <span class="token operator">=</span> <span class="token string">"Store and access data published by OwnTracks apps"</span><span class="token punctuation">;</span><br /> homepage <span class="token operator">=</span> <span class="token string">"https://github.com/owntracks/recorder"</span><span class="token punctuation">;</span><br /> license <span class="token operator">=</span> licenses<span class="token punctuation">.</span>gpl2<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Now that we have it packaged, let's setup our server with OwnTracks, again using nix. I'll try to explain how the same thing will be done on Debian/Ubuntu derivatives.</p>
<p>Firstly create a systemd service to keep OwnTracks recorder running. Something like below should be a good start. Just FYI, <code>personal</code> is my nur channel directly pulled from GitHub. We use <code>-S</code> specify the data directory and <code>--doc-root</code> to specify the basic frontend files that recorder serves on <code>/</code>. I also pass in <code>--port 0</code> to disable the MQTT part. You can also pass these in a config file if you want. With that, your server is up and ready to go.</p>
<pre class="language-nix"><code class="language-nix">systemd<span class="token punctuation">.</span>services<span class="token punctuation">.</span>owntracks <span class="token operator">=</span> <span class="token punctuation">{</span><br /> enable <span class="token operator">=</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /> description <span class="token operator">=</span> <span class="token string">"Store and access data published by OwnTracks apps"</span><span class="token punctuation">;</span><br /> wantedBy <span class="token operator">=</span> <span class="token punctuation">[</span> <span class="token string">"multi-user.target"</span> <span class="token punctuation">]</span><span class="token punctuation">;</span><br /><br /> serviceConfig <span class="token operator">=</span> <span class="token punctuation">{</span><br /> WorkingDirectory <span class="token operator">=</span> <span class="token string">"<span class="token interpolation"><span class="token antiquotation important">$</span><span class="token punctuation">{</span>personal<span class="token punctuation">.</span>otrecorder<span class="token punctuation">}</span></span>"</span><span class="token punctuation">;</span><br /> ExecStart <span class="token operator">=</span> <span class="token string">"<span class="token interpolation"><span class="token antiquotation important">$</span><span class="token punctuation">{</span>personal<span class="token punctuation">.</span>otrecorder<span class="token punctuation">}</span></span>/bin/ot-recorder -S /home/meain/owntracks-data --doc-root usr/share/ot-recorder --port 0"</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre>
<p><a href="https://github.com/owntracks/recorder/blob/master/etc/ot-recorder.service">Here</a> is a systemd service file for when using something like Ubuntu</p>
<h2 id="configuring-nginx" tabindex="-1">Configuring nginx <a class="direct-link" href="https://blog.meain.io/2022/self-hosted-location-history/#configuring-nginx">#</a></h2>
<p>We can directly expose it, but in my case I wanted put some restrictions. I wanted to only expose <code>/pub</code> (this is the endpoint where we push/publish updates) and also add basic auth. Below is what the config will look like. You can uncomment the commented section to expose <code>/</code> as well (this will give you a very basic frontend). This sets up the subdomain <code>tracking.example.com</code> to proxy request to OwnTracks recorder. One additional thing that you will have to do is to create a passwords file by running <code>sudo htpasswd -c /etc/nginx/htpasswd owntracks</code> . This will prompt you for a password and once done, it will save the hashed password to <code>/etc/nginx/htpasswd</code> with the user as <code>owntracks</code>.</p>
<pre class="language-nix"><code class="language-nix">services<span class="token punctuation">.</span>nginx<span class="token punctuation">.</span>virtualHosts<span class="token punctuation">.</span><span class="token string">"tracking.example.com"</span> <span class="token operator">=</span> <span class="token punctuation">{</span><br /> enableACME <span class="token operator">=</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /> forceSSL <span class="token operator">=</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /> <span class="token comment"># use to serve owntracks basic ui (not owntracks/frontend)</span><br /> <span class="token comment"># locations."/".extraConfig = ''</span><br /> <span class="token comment"># auth_basic "OwnTracks pub";</span><br /> <span class="token comment"># auth_basic_user_file /etc/nginx/htpasswd; # sudo htpasswd /etc/nginx/htpasswd owntracks</span><br /> <span class="token comment"># proxy_pass http://localhost:8083;</span><br /> <span class="token comment"># proxy_http_version 1.1;</span><br /> <span class="token comment"># proxy_set_header Host $host;</span><br /> <span class="token comment"># proxy_set_header X-Real-IP $remote_addr;</span><br /> <span class="token comment"># proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;</span><br /> <span class="token comment"># '';</span><br /> locations<span class="token punctuation">.</span><span class="token string">"/pub"</span><span class="token punctuation">.</span>extraConfig <span class="token operator">=</span> <span class="token string">''<br /> auth_basic "OwnTracks pub";<br /> auth_basic_user_file /etc/nginx/htpasswd;<br /> proxy_pass http://localhost:8083;<br /> proxy_http_version 1.1;<br /> proxy_set_header Host $host;<br /> proxy_set_header X-Real-IP $remote_addr;<br /> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;<br /> ''</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre>
<p>If you are using Ubuntu, you can just have what is there under <code>locations."/pub".extraConfig</code> in your nginx config under <code>location /pub</code> for your domain.</p>
<h2 id="setting-up-android-app" tabindex="-1">Setting up Android app <a class="direct-link" href="https://blog.meain.io/2022/self-hosted-location-history/#setting-up-android-app">#</a></h2>
<p>Well, this is the simplest part. Download the app from <a href="https://f-droid.org/packages/org.owntracks.android/">FDroid</a> or <a href="https://play.google.com/store/apps/details?id=org.owntracks.android&gl=US">Play Store</a>. There is a little bit of setup to make it aware of the server. Navigate to Preferences > Connection and setup the following:</p>
<ul>
<li>Set <code>Mode</code> as <code>HTTP</code> (Good ol' Hyper Text Transfer Protocol)</li>
<li>Set <code>Host</code> as <code>https://tracking.example.com/pub</code> (use your subdomain here)</li>
<li>Set <code>Identification.Username</code> as the username you set (<code>owntracks</code> if you just followed the blog)</li>
<li>Set <code>Identification.Password</code> as the password you set (<code>B6H8^aR%#%pxv8y3eXNQS%RBRy^Sve</code> if you want a good one)</li>
<li>Set <code>Identification.DeviceID</code> as the name of your device (<code>ElectricClock</code> if you want a good one)</li>
</ul>
<p>Well, with that you are all set to track yourself. Now to viewing the data.</p>
<h2 id="setting-up-the-frontend" tabindex="-1">Setting up the frontend <a class="direct-link" href="https://blog.meain.io/2022/self-hosted-location-history/#setting-up-the-frontend">#</a></h2>
<p>You can find the code for the frontend <a href="https://github.com/owntracks/frontend">here</a>. I personally did not want to setup a frontend on the VM. What I do instead is to tunnel directly to the recorder server via ssh and use it to view my data when I want. But it basically is just a Vue bundle which you can build to a static set of files and serve over anything. One thing you might wanna look into is setting the server address in <code>src/config.js</code> <a href="https://github.com/owntracks/frontend#configuration">ref</a>. You can find more instructions on how you can set it up on their <a href="https://github.com/owntracks/frontend#installation">GitHub</a>.</p>
<p><img src="https://blog.meain.io/img/owntracks-frontend.png" alt="" /></p>
<p><em>Again, not my screenshot :D</em></p>
<p>Aaaand you are done. I personally am happy with this setup for now, gives me both piece of mind as well as location history :D.</p>
EmacsConf 20222022-12-05T00:00:00Zhttps://blog.meain.io/2022/emacsconf-2022/<p>Hey! EmacsConf just concluded and it was a really awesome one this year. A lot of interesting talks. It was a lot of fun interacting with everyone during the event. For folks who where there, "hi again!".</p>
<p>If you have read my previous blogs, you might be already aware that I have been playing around a lot with <code>tree-sitter</code> in Emacs. And so, this year for EmacsConf I decided to talk about the same. It was my fist talk at EmacsConf and was really smooth experience :D. Thanks a lot to the organizers, mainly <a href="https://sachachua.com/">Sacha Chua</a> and <a href="http://zaeph.net/">Leo Vivier</a> for helping me through all of it. Also thanks to <a href="https://geeksocket.in/">Bhavin Gandhi</a> whom I've bounced off a lot of ideas.</p>
<blockquote>
<p>In case you are interested, you can checkout my talk at <a href="https://emacsconf.org/2022/talks/treesitter/">https://emacsconf.org/2022/talks/treesitter/</a> . The recording, transcript and Q&A should be available there. <em>The recording is available on <a href="https://www.youtube.com/watch?v=MZPR_SC9LzE">YouTube</a> as well if you wanna go that route.</em></p>
</blockquote>
<p>I'm still to catch up on all the talks, but from the ones that I have seen, these are the ones that I personally found useful:</p>
<h3 id="general" tabindex="-1">General <a class="direct-link" href="https://blog.meain.io/2022/emacsconf-2022/#general">#</a></h3>
<ul>
<li><a href="https://emacsconf.org/2022/talks/buddy/">The Emacs Buddy initiative</a>: Seems like a interesting way to get in touch with folks who have similar interests, I should probably consider signing up for this :D.</li>
<li><a href="https://emacsconf.org/2022/talks/meetups/">Attending and organizing Emacs meetups</a>: Lot of great insights into conducting Emacs meetups and guides on how one can attend/create one.</li>
<li><a href="https://emacsconf.org/2022/talks/survey/">Results of the 2022 Emacs Survey</a>: Great set of insights in to how different trends related to Emacs looks like. Looking forward to more analysis around the data.</li>
<li><a href="https://emacsconf.org/2022/talks/workflows/">Org workflows for developers</a>: While I don't personally use Org, the idea here seems pretty interesting.</li>
</ul>
<h3 id="development" tabindex="-1">Development <a class="direct-link" href="https://blog.meain.io/2022/emacsconf-2022/#development">#</a></h3>
<ul>
<li><a href="https://emacsconf.org/2022/talks/wayland/">Emacs should become a Wayland compositor</a>: This seems like a great next step after EXWM.</li>
<li><a href="https://emacsconf.org/2022/talks/rde/">rde Emacs introduction</a>: I've been following some of his development on <a href="https://www.youtube.com/@abcdw">YouTube</a>.</li>
<li><a href="https://emacsconf.org/2022/talks/detached/">Getting detached from Emacs</a>: Looks like a great alternative to run servers and other long running processes from Emacs.</li>
<li><a href="https://emacsconf.org/2022/talks/eshell/">Top 10 reasons why you should be using Eshell</a>: I've been trying to switch to eshell and this shows a lot of interesting features of eshell and how having lisp available improves a lot of things.</li>
<li><a href="https://emacsconf.org/2022/talks/workflows/">Emacs was async before async was cool</a>: Neat intro to working with network calls in Emacs in an async manner</li>
<li><a href="https://emacsconf.org/2022/talks/dbus/">The Wheels on D-Bus</a>: Loved the idea of being able to use D-Bus from within Emacs to interact with more of the system</li>
</ul>
What is in a modern code editor?2022-12-11T00:00:00Zhttps://blog.meain.io/2022/modern-text-editor/<h1 id="what-do-i-know%3F" tabindex="-1">What do I know? <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#what-do-i-know%3F">#</a></h1>
<p>Before we begin, let me give you an idea of where I am coming from and what my experiences are based on. Just to get it out of the way, I'm currently an Emacs user. That said, I have used a lot of text/code editors, all the way from Notepad to JetBrains IDEs(well, in the past) or even the Borland C++ editor which looked like this running on DOS.</p>
<p><img src="https://blog.meain.io/img/turbo-borland.png" alt="Screenshot of Turbo Borland editor" /></p>
<p>I still remember the day I learnt that you can type <code>sop</code> and hit <kbd>ctrl</kbd> + <kbd>space</kbd> in <a href="https://www.oracle.com/tools/technologies/netbeans-ide.html">Netbeans</a> to complete to <code>System.out.println</code>. This was back in my school days. I also vividly remember the <a href="https://www.codeblocks.org/">Code::Blocks</a> splash screen to this day. It was definitely a thing of beauty. Well, enough reminiscing on the past and the fact that I am getting old.</p>
<p>The code editors that I have spent most time with however are Sublime Text, Neovim, Emacs and VS Code. That said, I have used quite a few other editors as well over the years out of which, the ones that I thought were interesting are <a href="https://chrome.google.com/webstore/detail/zed-code-editor/pfmjnmeipppmcebplngmhfkleiinphhp?hl=en">zed</a> (not the <a href="https://zed.dev/">new one</a>, I haven't got my hands on that), <a href="https://kakoune.org/">kakoune</a>, <a href="https://helix-editor.com/">helix</a>, <a href="https://xi-editor.io/">xi</a>. I kinda also like watching <a href="https://github.com/bisqwit">bisqwit</a> use <a href="https://github.com/bisqwit/that_editor">That editor</a> or <a href="https://www.youtube.com/watch?v=NgmU9iy8P44">Casey</a> use <a href="https://4coder.net/">4coder</a>.</p>
<p>In this post, the idea is to convey what I think a modern text editor should be capable of doing and what things are generally used to power them. Now, without further <em>adieu</em>, let's get into the meat(or something else if you are vegan) of the blog.</p>
<h1 id="syntax-highlighting" tabindex="-1">Syntax highlighting <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#syntax-highlighting">#</a></h1>
<p>While <a href="https://youtu.be/aHm36-na4-4?t=612">not everyone needs syntax highlighting</a>, most people do. For the best part of text editor history, syntax highlighting was powered by a lot of hacky regexes. One significant improvement came with <a href="https://macromates.com/">TextMate</a> which introduced <a href="https://macromates.com/manual/en/language_grammars">TextMate Grammar</a> which is a more organized way of writing something to syntax highlight code but still relied on the old hacky system. But these days we have something much better. We have <a href="https://tree-sitter.github.io/tree-sitter/">Tree-sitter</a> which unlike using regexes actually parses your code and forms a syntax tree which you can query on top to figure out how to highlight your code. Tree-sitter is useful for a lot more than just syntax highlighting, but it makes syntax highlighting a lot more efficient. <a href="https://www.youtube.com/watch?v=Jes3bD6P0To">Here</a> is a great talk by the original author introducing tree-sitter.</p>
<blockquote>
<p>For those interested, I <a href="https://emacsconf.org/2022/talks/treesitter/">gave a talk</a> at <a href="https://emacsconf.org/">EmacsConf</a> this year about Tree-sitter.</p>
</blockquote>
<p>While this is an area which has mostly been figured out, we are still debating and coming up with interesting ideas around syntax highlighting like <a href="https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide">semantic highlighting</a> which lets you highlight code based on semantics in addition to just syntax.</p>
<p><a href="https://www.youtube.com/watch?v=l1b7Da2DnPo">Building My Own Clojure Tools - Nikita Prokopov (tonsky)</a> is a pretty good talk which discussion on syntax highlighting along with other things like font and indentation. There is also an <a href="https://youtu.be/oka4wcsrg0c">interesting talk by Damian Conway</a> which gave me the idea of using backgrounds for showing errors. I also want to introduce you to some works by Nicolas P. Rougier around Emacs. <a href="https://www.youtube.com/watch?v=7OTe26RZH9A">Talk: On design of text editors by Nicolas P. Rougier</a> and <a href="https://arxiv.org/abs/2008.06030">Paper: On design of text editors by Nicolas P. Rougier</a>.</p>
<p>Here is how my Emacs looks like. I personally use a minimal theme highlighting just the function names in declarations, comments and strings. For those interested, you can find my Emacs theme <a href="https://github.com/meain/dotfiles/blob/master/emacs/.config/emacs/hima-theme.el">hima</a> in my <a href="https://github.com/meain/dotfiles">dotfiles</a>.</p>
<p><img src="https://blog.meain.io/img/emacs-syntax-highlight.png" alt="Screenshot of my Emacs session editing go code" /></p>
<h1 id="browsing-and-picking-files" tabindex="-1">Browsing and picking files <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#browsing-and-picking-files">#</a></h1>
<p>File pickers, come in various forms. You have your classic side pane, side pane with tree, fuzzy finding, inline tree. At the very least, most editors comes with some way to browse files, for example <a href="https://www.emacswiki.org/emacs/DiredMode">dired</a> in Emacs and <a href="https://vimhelp.org/pi_netrw.txt.html#netrw">netrw</a> in vim or the file tree side panel in VS Code.</p>
<p>While file browsers are useful, I find some way to select a file using some kind of string matching to be a much better option. Most popular editors also do support this with Sublime Text popularizing this idea IIRC. In rudimentary versions, these are done using string matching on all the files available in a project, of one level up is fuzzy matching.</p>
<p>Again, Emacs does this much better using the <a href="https://github.com/oantolin/orderless">orderless</a> package which provides one with insanely powerful filtering abilities. In the following video you can see that I first narrow it down to just go files using the regex <code>.go$</code>, then search for <code>utils</code>, then filter out things which have the word test in them using <code>!test</code> and finally look for patters which have three words starting with p using <code>,ppp</code>.</p>
<p><video src="https://blog.meain.io/img/emacs-file-picker.mp4" autoplay="" controls="" loop=""></video></p>
<p>Another benefit of Emacs is that you can <a href="https://emacsrocks.com/e16.html">edit the dired buffers as if it was just text</a> with the full power of macros or any other elisp functions and they will reflect in the filesystem.</p>
<p>Now to give some credit to folks at Neovim, they have <a href="https://github.com/nvim-telescope/telescope.nvim">Telescope</a> which gives you a live preview of the buffer. Emacs lets you do that too, but yeah. I used to previously do this by integrating <a href="https://github.com/junegunn/fzf">fzf</a> but the colors did not match the colorscheme of the editor.</p>
<p>Another neat tool in this area is <a href="https://github.com/Canop/broot">broot</a>, which lets you fuzzy find with the context of folder hierarchy as a tree.</p>
<h1 id="searching-through-the-codebase" tabindex="-1">Searching through the codebase <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#searching-through-the-codebase">#</a></h1>
<p>Back in the good old days, we just had grep, then came <a href="https://beyondgrep.com/">ack</a>, then <a href="https://github.com/ggreer/the_silver_searcher">ag</a> and now everyone seems to have settled on to <a href="https://github.com/BurntSushi/ripgrep">rg</a>. These are all cli tools which lets you do grep like things, but most good editors have some way to do these kinds of searches from within them and/or integrate these tools into their editors.</p>
<p>In most cases when you are staring off with a new codebase or just wanna find where that one pesky little thing is coming from grep is always your best friend. If I am not mistaken VS Code also uses rg to project wide search. So is the case for Emacs and Neovim though in the latter two you can just swap out the grep engine easily.</p>
<p>A great feature of Emacs is that you can turn the grep results buffer into an editable buffer and <a href="https://www.youtube.com/watch?v=OcR-Ke2CiPo">edit across all the results</a> from multiple files.</p>
<p>Here is a screenshot of <a href="https://github.com/dajva/rg.el">rg.el</a> in Emacs.
<img src="https://blog.meain.io/img/rg-el.png" alt="Screenshot of rg.el in Emacs" /></p>
<h1 id="language-intelligence" tabindex="-1">Language intelligence <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#language-intelligence">#</a></h1>
<p>These are things like intelligent autocompletion, go to definition, find references, rename, refactor etc. It is one of those features that used to be restricted to just big IDEs, but ever since <a href="https://langserver.org/">LSP</a> has been a thing, any editor with an lsp client can get a lot of the language intelligence. While I'm not a big fan of Microsoft, you still have to give them credit for introducing the world to the idea LSP though VS Code.</p>
<p>While we used to have <a href="https://ctags.io/">ctags</a> or <a href="https://www.gnu.org/software/global/">GNU Global</a> it was just the basics jumping around and was at times a bit clunky and not real time in updating the lookup files. But now with lsp, you get most of the important bells and whistles of IDEs with none(only a little) of that bloat.</p>
<p>Beyond just navigating and refactoring source code, lsp servers have evolved into a way for someone to have an editor agnostic way to create servers which will let users do interesting things. One such example would be <a href="https://github.com/joe-re/sql-language-server">joe-re/sql-language-server</a> which can connect to your db and show docs, generate completion candidates, run sql queries etc. Having an lsp for <a href="https://github.com/oncomouse/citation-langserver">bibtex</a>, <a href="https://github.com/latex-lsp/texlab">LaTeX</a> or <a href="https://github.com/ejgallego/coq-lsp">coq</a> is also pretty useful. You can use lsp to validate and provide completions and docs in your config files using <a href="https://github.com/microsoft/vscode/tree/main/extensions/json-language-features/server">json-language-server</a> and <a href="https://github.com/redhat-developer/yaml-language-server">yaml-language-server</a> along with <a href="https://github.com/redhat-developer/yaml-language-server#more-examples-of-schema-association">json schema</a>. Having <a href="https://about.sourcegraph.com/blog/sourcegraph-code-intelligence-and-the-language-server-protocol">sourcegraph</a> being pluggable as an lsp server was also pretty interesting. There is also <a href="https://github.com/mattn/efm-langserver">efm-langserver</a> which lets you create language servers from binaries like linters.</p>
<p>Here is a gif of lsp providing autocompletions in Sublime Text.
<img src="https://blog.meain.io/img/sublime-lsp-autocomplete.gif" alt="GIF of lsp providing autocompletions in Sublime Text" /></p>
<p>Kinda unrelated, but Raph Levien had some interesting ideas early on when working on Xi on how external tools should interact with an editor. You can see the recording <a href="https://www.youtube.com/watch?v=4FbQre9VQLI">here</a>.</p>
<h1 id="debugging-capabilities" tabindex="-1">Debugging capabilities <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#debugging-capabilities">#</a></h1>
<p>Talking more about the things that only IDEs(well, Emacs too) used to have, next is debugging. Debug Adapter Protocol (DAP) is another thing which you have to give Microsoft credit for. DAP helps bring the ability to debug code in your editor using a server started by language specific tool which you can connect to and work with.</p>
<p>This is probably something that I have taken least advantage of from all the things that Emacs can do. I still do most of my debugging outside the editor in a terminal. I started my debugging journey with <a href="https://www.sourceware.org/gdb/"><code>gdb</code></a>, then used a lot of <a href="https://github.com/gotcha/ipdb"><code>ipdb</code></a> or <a href="https://github.com/inducer/pudb"><code>pudb</code></a> and nowadays a lot of <a href="https://github.com/go-delve/delve"><code>dlv</code></a>. I am trying to fix this problem of mine where I don't use the editor for debugging by building an Emacs plugin to do debugging using dap(there is already <a href="https://github.com/emacs-lsp/dap-mode">dap-mode</a>) called <a href="https://twitter.com/meain_/status/1249570644032761857">dap-dance</a> which is still unreleased and might stay that way forever.</p>
<p><em>I really found <a href="https://www.youtube.com/watch?v=I845O57ZSy4&t=3410s">this</a> interaction between Lex Fridman and John Carmack around debuggers pretty interesting.</em></p>
<p>Below is a screenshot of <a href="https://github.com/rcarriga/nvim-dap-ui">nvim-dap-ui</a>. You can see that it has most of the kind of things that you need from a debugger. Your watch window, backtrace, locals; it's all there.</p>
<p><img src="https://blog.meain.io/img/nvim-dap-ui.png" alt="Screenshot of nvim-dap-ui" /></p>
<h1 id="ability-to-run-linters-and-show-errors" tabindex="-1">Ability to run linters and show errors <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#ability-to-run-linters-and-show-errors">#</a></h1>
<p>Linters are one of those things which are close to my heart. Ever since I realized I can use linters to look into my code to find dumb shit I do, I have been a fan of them.</p>
<p>I am from that era where you had to save a file, then wait for vim to run linters on your code before you could do anything. <em>Kids these days have it so easy, they have linters continuously running while they edit.</em> I remember when <a href="https://github.com/dense-analysis/ale">ale</a> came out for vim being so excited that I don't have to wait after every save to get my lint errors popping up in vim. Of course, Emacs already had similar things at that point but I was an Vim person back then.</p>
<p>Most editors these days have the ability to run linters on code in the current active buffer even before you save it. It can take the unsaved source as it is in the editor, pipe it to the linter and get the results back and render them in the text editor as well as provide ways to navigate between different errors. Most of them end up drawing those squiggly red underlines where the error occurred and you can go to them and see what the error is.</p>
<p>Here is a screenshot of <a href="https://www.flycheck.org/en/latest/">Flycheck</a> in Emacs showing lint messages:</p>
<p><img src="https://blog.meain.io/img/flycheck-annotated.webp" alt="Image of Flycheck in Emacs showing off its features" /></p>
<p>It is not just code, while writing prose you can have your editor integrate spelling and grammar checking tools as linters and they can vet your prose.</p>
<h1 id="formatting-code" tabindex="-1">Formatting code <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#formatting-code">#</a></h1>
<p>Next one on my favorites list after linters is auto formatters. The only reason this is second is because I might be able to make a rudimentary formatter on my own, but need someone else to make a linter. Having an auto formatter is a godsend in most cases.</p>
<p>You write the code, they take care of making it look pretty or more importantly consistent with everyone else's code. I believe Go was the first language to go all in on auto formatting, but most other languages followed suit. No more arguing and worrying about where to put that <code>{</code> or if we should put a space before <code>+</code>. Right after <a href="https://go.dev/blog/gofmt"><code>gofmt</code></a> became popular, we had <a href="https://prettier.io/">prettier</a> for javascript, <a href="https://github.com/psf/black">black</a> for python, <a href="https://github.com/rust-lang/rustfmt"><code>rustfmt</code></a> for rust and a lot of others.</p>
<p>We even have formatters for sql and prettier has a formatter for markdown as well which can nicely format markdown tables. Being able to format generated html, css or json with just one keystroke is really useful.</p>
<p><a href="https://youtu.be/sln-gJaURzk?t=1735">Here</a> is a really interesting discussion about go fmt with the core go team. I don't know if it is in this video or somewhere else, but I remember a statement from Rob Pike which goes along the lines of "Nobody likes the way gofmt formats their code, but everyone like that it formats their code". I personally found it pretty relatable. You might not like the way the formatter formats the code, but there is a lot of advantages that you get from everyone's code looking the same. It is a lot more easy to parse things when you are reading code plus you start to get this gut feeling that something doesn't look right just from the look of it.</p>
<p>These days, most languages have autoformatters and any good editor should have the ability to use them to format the users code. Most of them also have a setting which you can enable to automatically format your code before you save them which could come in handy if everyone in your team has already agreed to using a specific formatter.</p>
<h1 id="terminal-emulator" tabindex="-1">Terminal emulator <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#terminal-emulator">#</a></h1>
<p>While you can hack around an connect your terminal emulator of your choice to talk to your editor, it is better to have some way of emulating a terminal in your editor. Emacs is so good at it that it has 6 different options, <a href="https://www.gnu.org/software/emacs/manual/html_mono/eshell.html">eshell</a>, <a href="https://www.gnu.org/software/emacs/manual/html_node/emacs/Term-Mode.html">term</a>, <a href="https://www.gnu.org/software/emacs/manual/html_node/emacs/Shell.html">shell</a>, <a href="https://www.emacswiki.org/emacs/AnsiTerm">ansi-term</a>, <a href="https://github.com/akermu/emacs-libvterm">vterm</a> and <a href="https://codeberg.org/akib/emacs-eat">eat</a>.</p>
<p>You will always run into cases where you would like to run commands while working on code. You can, most of the time get away with external terminals but having it as a component of your text editor makes it much easier to integrate your code and terminal. For example navigating to a error in the output you got in a compilation using the line and column number mentioned is much easier if the terminal emulator is part of your editor. When I used to vim/neovim, I was mostly getting away with using it in tmux and writing integrations between tmux and vim to get similar things done. These days, neovim and vim has built in terminal emulators as well.</p>
<p>Here is how integrated terminal looks like in <a href="https://github.com/VSCodium/vscodium">Codium</a>.</p>
<p><img src="https://blog.meain.io/img/codium-integrated-terminal.png" alt="" /></p>
<p>That said, you don't always need a terminal emulator. <a href="https://kakoune.org/">Kakoune</a> can get away without one as they expect you to be either running within a tiling window manager or tmux context and expect you to have some other way to using the shell.</p>
<h1 id="git%2Fvcs-integration" tabindex="-1">Git/VCS integration <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#git%2Fvcs-integration">#</a></h1>
<p>Yet another things which you can get away without integrating directly into the editor, but really helpful if you do have it. I am not just talking about the ability to view diff and commit code, but instead things like <a href="https://github.com/airblade/vim-gitgutter">indicators</a> to show that you edited something, helping <a href="https://www.youtube.com/watch?v=f_xMuOK0FuQ">resolve merge conflicts</a> or viewing <a href="https://github.com/gitkraken/vscode-gitlens">who and how lines changed</a> or even <a href="https://github.atom.io/">interacting with forges</a>(<a href="https://github.blog/2022-06-08-sunsetting-atom/">RIP Atom</a>). For those who have not heard about it, <a href="https://magit.vc/">Magit</a> in Emacs is just freaking bonkers. You can see a video about it <a href="https://www.youtube.com/watch?v=vQO7F2Q9DwA">here</a>.</p>
<p>Here is a screenshot of Magit in Emacs.
<img src="https://blog.meain.io/img/magit.png" alt="" /></p>
<h1 id="configuration-language-and-plugin-system" tabindex="-1">Configuration language and plugin system <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#configuration-language-and-plugin-system">#</a></h1>
<p>This is probably one of the most important or one of the least important things based on who you ask. For me personally, I don't think I'll be using a text editor that cannot be configured to my liking. I am not talking about having a config file where you can edit keybindings, but rather a system which will let me tweak the behavior of the system by writing code(ideally while it is running). This lets you make the editor your own, make it work the way you want instead of adapting to how it wants you to work.</p>
<p>In this regards, Emacs is probably the best you can get as of now IMO. Emacs is built mostly using elisp(except for a C base to build elisp and some core perf critical pieces). This gives you a lot of <a href="https://youtu.be/sBhQ2NIcrLQ?t=74">configurability</a> and this knowledge can apply to working on core emacs as well. If you didn't know even a simple action of inserting a character in a buffer is an elisp function which you can edit just like any other thing.</p>
<h1 id="now-for-a-few-more-entries" tabindex="-1">Now for a few more entries <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#now-for-a-few-more-entries">#</a></h1>
<p>Here are some less newsworthy but still important ones.</p>
<ul>
<li>Support for snippets and templating: I know a lot of people don't use it, but it can help simplify in building out boilerplate stuff</li>
<li>Test runners: Being able to do things like "run current test" is really useful when you are working off of a test</li>
<li>Window management: Any good text editor should be able to manage multiple windows and switch between them</li>
<li>Code folding: Another thing which some people swear by, but lot other don't even care</li>
<li>Undo system: This is something that you don't think about when it works, but annoys when it does not. Some editors have ways to have <a href="https://jovicailic.org/2017/04/vim-persistent-undo/">infinite undos</a> or have <a href="https://github.com/apchamberlain/undo-tree.el">tree view of undos</a>.</li>
<li>Understanding indentation: Try using something that does not know when to indent, it is painful.</li>
</ul>
<h1 id="and-even-more-things" tabindex="-1">And even more things <a class="direct-link" href="https://blog.meain.io/2022/modern-text-editor/#and-even-more-things">#</a></h1>
<p>Now we are getting into my personal opinions(as if the rest was just objective truth).</p>
<ul>
<li>Looking up docs: While you can look up docs outside the editor in your browser, it makes it much faster and easier to look up docs within your editor</li>
<li>Working on remote system: This is kinda niece, but being able to work on remote systems just like it is your local machine is really useful.</li>
<li>Modeline: A modeline or some way to view details about the current file like name, error count, line number etc can come in handy.</li>
<li>Good font rendering and teeming capabilities: Being able to render fonts properly, have features like ligatures, good theming api etc are all important to making it look pretty.</li>
<li>Vim mode: Well, although I love Emacs, if it did not have vim mode in the form of <a href="https://github.com/emacs-evil/evil">evil-mode</a> I probably would not use it.</li>
</ul>
<p>And I'll stop here. I started this blog thinking I would talk about just lsp, dap and tree-sitter but half way pivoted to writing about all the things that I think should be in a modern code editor. Also, if at any point, it sounded like Emacs solves all your problems, it is because it does. Just kidding; use whatever you are comfortable with. Everything has its problems, but don't be afraid to try other things.</p>
<blockquote>
<p>There are some interesting discussions over on <a href="https://news.ycombinator.com/item?id=33950658#33953538">HackerNews</a> and <a href="https://lobste.rs/s/pmieoo/what_is_modern_code_editor">Lobsters</a> in case you want to check them out.</p>
</blockquote>
Releasing scopeline.el2023-03-12T00:00:00Zhttps://blog.meain.io/2023/releasing-scopeline-el/<p>Hey, this is a quick announcement about my new Emacs plugin, <a href="https://github.com/meain/scopeline.el">scopeline.el</a>. You can find it <a href="https://github.com/meain/scopeline.el">here</a>.</p>
<p>What is does is something pretty simple, but IMO pretty useful. It shows you what every closing delimiter is actually closing. When you have a large file and you are the end of a function, you might have a lot of closing brackets, one for the function, one for a loop, one for that conditional you started and one for that switch statement. Also, which function is this closing bracket for again? Answers for all of these question is provided by scopeline.el. It uses tree-sitter under the hood to parse the code and figure out what we are looking at and makes use of overlays to display it. <em>It works with <code>elisp-tree-sitter</code> as well as builtin <code>treesit</code> package.</em></p>
<p>In the below screenshot you can see that we get information about the function nesting that is going on and of that for loop. While it might seem kinda unnecessary in this image, imagine this information when you are not able to see that start of the function. This also comes in really handy for yaml files.</p>
<p><img src="https://user-images.githubusercontent.com/14259816/208631769-052ac0ab-44df-4949-8f2f-3ef43e249f65.png" alt="Screenshot of Emacs frame with scopeline.el enabled" /></p>
<p>You can enable it like blow:</p>
<pre class="language-emacs-lisp"><code class="language-emacs-lisp"><span class="token punctuation">(</span><span class="token keyword">use-package</span> scopeline<br /> <span class="token lisp-property property">:enable</span> <span class="token boolean">t</span><br /> <span class="token lisp-property property">:after</span> tree-sitter<br /> <span class="token lisp-property property">:config</span> <span class="token punctuation">(</span><span class="token car">add-hook</span> <span class="token quoted-symbol variable symbol">'tree-sitter-mode-hook</span> <span class="token quoted-symbol variable symbol">#'scopeline-mode</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment">; or use :hook</span></code></pre>
<p>This will set it up, and enable for all modes that tree-sitter is enabled for.</p>
<p>There are few customizations that you can do. Details about that can be found at in the <a href="https://github.com/meain/scopeline.el#configuration">README</a> file.</p>
Splitting and joining using tree-sitter2023-03-15T00:00:00Zhttps://blog.meain.io/2023/split-join-tree-sitter/<blockquote>
<p>EDIT: I've since updated the code to support leading commas. You can find that version <a href="https://github.com/meain/dotfiles/commit/1a63dc731d49e4045c76cf5afc07f4adffce3850">here</a></p>
</blockquote>
<p>Another day, another blog on tree-sitter. Not sure if you have read any of my previous blogs on tree-sitter, <a href="https://blog.meain.io/2021/releasing-evil-textobj-tree-sitter">but</a> <a href="https://blog.meain.io/2021/intelligent-snippets-treesitter">I</a> <a href="https://blog.meain.io/2022/more-treesitter-emacs">have</a> <a href="https://blog.meain.io/2022/navigating-config-files-using-tree-sitter">a</a> <a href="https://blog.meain.io/2022/emacsconf-2022">few</a> <a href="https://blog.meain.io/2022/modern-text-editor">of</a> <a href="https://blog.meain.io/2023/releasing-scopeline-el">them</a>.</p>
<p>First, let me show you what this one is about. Here is a recording of the code in action.</p>
<p><video src="https://meain.io/blog-videos/videos/tree-surgeon-split-join.mp4" controls="">Video of the code in action splitting and joining rust and json examples</video></p>
<p>You can use this to split/join arguments in a function definition or call into multiple lines or split/join items in a json doc.</p>
<p>You might think it is a simple job that a "split by comma" can do, but you would be wrong. What if you have a <code>,</code> in a string argument or if one of the arguments is a function that has arguments inside it or a list within a list. What if the user just puts a lot of empty newlines in between? The <s>package</s> function deals with all of this by letting things built by smarter people (tree-sitter) deal with it. We just ask it to get us the list of arguments which we arrange how it supposed to be.</p>
<p>You can find the code for this in my <a href="https://github.com/meain/dotfiles/blob/1d81592e242e0617cdb081c3bc37ebc979d4eedb/emacs/.config/emacs/tree-surgeon-split-join.el">dotfiles</a>. Once you have this file in your Emacs, just make sure to load it and then you can call <a href="https://github.com/meain/dotfiles/blob/1d81592e242e0617cdb081c3bc37ebc979d4eedb/emacs/.config/emacs/tree-surgeon-split-join.el#L41"><code>tree-surgeon-split-join</code></a> and it will alternate between single line and multiline. <em>As of now it only support go, rust and json, but that is because that is what I need. Mostly just go as the formatter does not split long lines.</em> That said, it should be easy (just a line) to add support for more languages or more things that you can split and join. You can add them in <a href="https://github.com/meain/dotfiles/blob/1d81592e242e0617cdb081c3bc37ebc979d4eedb/emacs/.config/emacs/tree-surgeon-split-join.el#L28"><code>tree-surgeon-split-join-settings</code></a> if you understand what is going on, or you can reach out to me if you need any help.</p>
<p>Let me explain how it works so that you can edit this for your use case. We'll walk through an example of go buffer. In case of go, the <a href="https://github.com/meain/dotfiles/blob/1d81592e242e0617cdb081c3bc37ebc979d4eedb/emacs/.config/emacs/tree-surgeon-split-join.el#L32">config</a> is <code>((argument_list parameter_list) . t)))</code> which means we can split and join <code>argument_list</code> and <code>parameter_list</code>. The <code>t</code> indicates that it requires a trailing comma. First we look for all the nodes that we are in with these as the type (<a href="https://github.com/meain/dotfiles/blob/1d81592e242e0617cdb081c3bc37ebc979d4eedb/emacs/.config/emacs/tree-surgeon-split-join.el#L50">ref</a>). Once we have that, we find the one that we are closest to and <a href="https://github.com/meain/dotfiles/blob/1d81592e242e0617cdb081c3bc37ebc979d4eedb/emacs/.config/emacs/tree-surgeon-split-join.el#L52">find the direct children</a> of it. Once we have that, text is extracted from these and used as the basis for splitting or joining them (<a href="https://github.com/meain/dotfiles/blob/1d81592e242e0617cdb081c3bc37ebc979d4eedb/emacs/.config/emacs/tree-surgeon-split-join.el#L56">ref</a>). This gets used in the body of the let expressions. When joining, we can create a <a href="https://github.com/meain/dotfiles/blob/1d81592e242e0617cdb081c3bc37ebc979d4eedb/emacs/.config/emacs/tree-surgeon-split-join.el#L57">single line with all the items joined</a> and when splitting we create items <a href="https://github.com/meain/dotfiles/blob/1d81592e242e0617cdb081c3bc37ebc979d4eedb/emacs/.config/emacs/tree-surgeon-split-join.el#L65">line by line</a> adding proper indentation. And that is pretty much it.</p>
<hr />
<p>I'm planning to move this to a separate plugin, and thus the name <code>tree-surgeon</code>, but that will need more time plus I want to make sure I figure out the scope of the plugin. The idea is that it will be a generic set of tree-sitter based buffer operations that you can do. I have a few other in my config that I would can extract out as well already.</p>
What is in that .git directory?2023-10-06T00:00:00Zhttps://blog.meain.io/2023/what-is-in-dot-git/<p>Well, I think most of you reading this blog use <code>git</code> more or less on a daily basis, but have you ever looked into what is in the <code>.git</code> folder that git creates? Let's explore it together and understand what is going on in there.</p>
<p><em>This is a blog version of a talk that I recently gave. Unfortunately I can't link to the recording :(.</em></p>
<blockquote>
<p><code>git</code> at a basic level is just a bunch of text files linked to each other by filenames.</p>
</blockquote>
<h1 id="let's-start-init" tabindex="-1">Let's <s>start</s> init <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#let's-start-init">#</a></h1>
<p>As you all know, we start our git journey with a <code>git init</code>. This gives the message that we all are probably used to by now, especially if you start and abandon a lot of side projects.</p>
<pre><code>Initialized empty Git repository in /home/meain/dev/src/git-talk/.git/
</code></pre>
<p>Let's look at what is in the <code>.git</code> repo as of now.</p>
<pre><code>$ tree .git
.git
├── config
├── HEAD
├── hooks
│ └── prepare-commit-msg.msample
├── objects
│ ├── info
│ └── pack
└── refs
├── heads
└── tags
</code></pre>
<p>It seems to create a bunch of files and folders. What are all these? Let's go over them one by one.</p>
<ul>
<li><code>config</code> is a text file that contains your git configuration for the current repo. If you look into it, you would see some basic settings for you repo like the author, filemode etc.</li>
<li><code>HEAD</code> contains the current head of the repo. Depending on what you have set your "default" branch to be, it will be <code>refs/heads/master</code> or <code>refs/heads/main</code> or whatever else you had set to. As you might have guessed, this is pointing to <code>refs/heads</code> folder that you can see below, and into a file called <code>master</code> which does not exist as of now. This file <code>master</code> will only show up after you make your first commit.</li>
<li><code>hooks</code> contain any scripts that can be run before/after git does anything. I have written another blog <a href="https://blog.meain.io/2019/making-sure-you-wont-commit-conflict-markers/">here</a> which goes a bit more into how git hooks work.</li>
<li><code>objects</code> contains the git objects, ie the data about the files, commits etc in your repo. We will go in depth into this in this blog.</li>
<li><code>refs</code> as we previously mentioned, stores references(pointers). <code>refs/heads</code> contains pointers to branches and <code>refs/tags</code> contains pointers to tags. We will go into what these files look like soon.</li>
</ul>
<h1 id="now-we-add-a-file" tabindex="-1">Now we add a file <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#now-we-add-a-file">#</a></h1>
<p>Now that you have an idea of what the initial set of files in <code>.git</code> is, let's perform the first action that adds something into the <code>.git</code> directory. Let's create a file and add it(we are not committing it yet).</p>
<pre class="language-bash"><code class="language-bash"><span class="token builtin class-name">echo</span> <span class="token string">'meain.io'</span> <span class="token operator">></span> <span class="token function">file</span><br /><span class="token function">git</span> <span class="token function">add</span> <span class="token function">file</span></code></pre>
<p>This does the following:</p>
<pre class="language-diff"><code class="language-diff"><span class="token coord">--- init 2024-07-02 15:14:00.584674816 +0530</span><br /><span class="token coord">+++ add 2023-07-02 15:13:53.869525054 +0530</span><br /><span class="token coord">@@ -3,7 +3,10 @@</span><br /><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">├── HEAD<br /></span><span class="token prefix unchanged"> </span><span class="token line">├── hooks<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ └── prepare-commit-msg.msample<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">├── index<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">├── objects<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">│ ├── 4c<br /></span><span class="token prefix inserted">+</span><span class="token line">│ │ └── 5b58f323d7b459664b5d3fb9587048bb0296de<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">│ ├── info<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ └── pack<br /></span><span class="token prefix unchanged"> </span><span class="token line">└── refs</span></span></code></pre>
<p>This causes two main changes as you can see. The first thing it modifies is the <code>index</code> file. The <a href="https://git-scm.com/docs/index-format">index</a> is what stores the information about what is currently staged. This is used to signify that the file named <code>file</code> has been added to the index.</p>
<p>The second and more important change is the addition of a new folder <code>objects/4c</code> and a file <code>5b58f323d7b459664b5d3fb9587048bb0296de</code> inside it.</p>
<h2 id="but-what-is-in-that-file%3F" tabindex="-1">But what is in that file? <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#but-what-is-in-that-file%3F">#</a></h2>
<p>Here is where we get into the details of how <code>git</code> stores things. Let's start with looking at what kind of data is present in that.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">file</span> .git/objects/4c/5b58f323d7b459664b5d3fb9587048bb0296de<br />.git/objects/4c/5b58f323d7b459664b5d3fb9587048bb0296de: zlib compressed data</code></pre>
<p>Hmm, but what is the zlib compressed data?</p>
<pre class="language-bash"><code class="language-bash">$ zlib-flate <span class="token parameter variable">-uncompress</span> <span class="token operator"><</span>.git/objects/4c/5b58f323d7b459664b5d3fb9587048bb0296de<br />blob <span class="token number">9</span><span class="token punctuation">\</span>0meain.io</code></pre>
<p>Looks like it contains the type, size and data of the file named <code>file</code> that we did a <code>git add</code> on. In this case, the data says that it is a <code>blob</code> of size <code>9</code> and the content is <code>meain.io</code>.</p>
<h2 id="ok%2C-but-what-is-with-that-filename%3F" tabindex="-1">OK, but what is with that filename? <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#ok%2C-but-what-is-with-that-filename%3F">#</a></h2>
<p>Well, good question. It comes from the sha1 of the content. If you take the zlib compressed data and pipe it through <code>sha1sum</code>, you get the filename.</p>
<pre class="language-bash"><code class="language-bash">$ zlib-flate <span class="token parameter variable">-uncompress</span> <span class="token operator"><</span>.git/objects/4c/5b58f323d7b459664b5d3fb9587048bb0296de<span class="token operator">|</span>sha1sum<br />4c5b58f323d7b459664b5d3fb9587048bb0296de</code></pre>
<p><code>git</code> takes the sha1 of the content to be written, takes the first two characters, in this case <code>4c</code>, creates a folder and then uses the rest of it as the filename. <code>git</code> creates folders from the first two chars to make sure we don't have too many files under the single <code>objects</code> folder.</p>
<h2 id="say-hello-to-git-cat-file" tabindex="-1">Say hello to <code>git cat-file</code> <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#say-hello-to-git-cat-file">#</a></h2>
<p>In fact, since this is one of the more important parts of git, git also has a plumbing command to view the content of an object. You can use <code>git cat-file</code> with <code>-t</code> for type, <code>-s</code> for size and <code>-p</code> for content.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">git</span> cat-file <span class="token parameter variable">-t</span> 4c5b58f323d7b459664b5d3fb9587048bb0296de<br />blob<br /><br />$ <span class="token function">git</span> cat-file <span class="token parameter variable">-s</span> 4c5b58f323d7b459664b5d3fb9587048bb0296de<br /><span class="token number">9</span><br /><br />$ <span class="token function">git</span> cat-file <span class="token parameter variable">-p</span> 4c5b58f323d7b459664b5d3fb9587048bb0296de<br />meain.io</code></pre>
<h1 id="let's-commit" tabindex="-1">Let's commit <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#let's-commit">#</a></h1>
<p>Now that we know what changes when we add a file, let's take this to the next level by committing.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">git</span> commit <span class="token parameter variable">-m</span> <span class="token string">'Initial commit'</span><br /><span class="token punctuation">[</span>master <span class="token punctuation">(</span>root-commit<span class="token punctuation">)</span> 4c201df<span class="token punctuation">]</span> Initial commit<br /> <span class="token number">1</span> <span class="token function">file</span> changed, <span class="token number">1</span> insertion<span class="token punctuation">(</span>+<span class="token punctuation">)</span><br /> create mode <span class="token number">100644</span> <span class="token function">file</span></code></pre>
<p>Here is what changed:</p>
<pre class="language-diff"><code class="language-diff"><span class="token coord">--- init 2024-07-02 15:14:00.584674816 +0530</span><br /><span class="token coord">+++ commit 2023-07-02 15:33:28.536144046 +0530</span><br /><span class="token coord">@@ -1,11 +1,25 @@</span><br /><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">.git<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">├── COMMIT_EDITMSG<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">├── config<br /></span><span class="token prefix unchanged"> </span><span class="token line">├── HEAD<br /></span><span class="token prefix unchanged"> </span><span class="token line">├── hooks<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ └── prepare-commit-msg.msample<br /></span><span class="token prefix unchanged"> </span><span class="token line">├── index<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">├── logs<br /></span><span class="token prefix inserted">+</span><span class="token line">│ ├── HEAD<br /></span><span class="token prefix inserted">+</span><span class="token line">│ └── refs<br /></span><span class="token prefix inserted">+</span><span class="token line">│ └── heads<br /></span><span class="token prefix inserted">+</span><span class="token line">│ └── master<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">├── objects<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">│ ├── 3c<br /></span><span class="token prefix inserted">+</span><span class="token line">│ │ └── 201df6a1c4d4c87177e30e93be1df8bfe2fe19<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">│ ├── 4c<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ │ └── 5b58f323d7b459664b5d3fb9587048bb0296de<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">│ ├── 62<br /></span><span class="token prefix inserted">+</span><span class="token line">│ │ └── 901ec0eca9faceb8fe0a9870b9b6cde75a9545<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">│ ├── info<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ └── pack<br /></span><span class="token prefix unchanged"> </span><span class="token line">└── refs<br /></span><span class="token prefix unchanged"> </span><span class="token line"> ├── heads<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> │ └── master<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> └── tags</span></span></code></pre>
<p>Woah, looks like there are a bunch of changes. Let's walk through them one by one. The first one is a new file <code>COMMIT_EDITMSG</code>. As the name might suggest, it contains the (last) commit message.</p>
<p><em>If you where to run the <code>git commit</code> command without the <code>-m</code> flag, the way <code>git</code> gets a commit message is to open an editor with the <code>COMMIT_EDITMSG</code> file to let the user edit the commit message and once the user has updated it and exited the editor, <code>git</code> uses the contents of the file as the commit message.</em></p>
<p>It also added a whole new folder <code>logs</code>. This is a way for git to log all the commits changes in a repo. You will be able to see the changes in commits for all refs and <code>HEAD</code> here.</p>
<p>The <code>object</code> dir also got some changes, but I want you to first look into the <code>refs/heads</code> directory where we now have the file <code>master</code>. This as you might have guessed is the reference to the <code>master</code> branch. Let's see what is in it.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">cat</span> refs/heads/master<br />3c201df6a1c4d4c87177e30e93be1df8bfe2fe19</code></pre>
<p>Looks like it is pointing to one of the new objects. We know how to look at objects, let's do that.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">git</span> cat-file <span class="token parameter variable">-t</span> 3c201df6a1c4d4c87177e30e93be1df8bfe2fe19<br />commit<br /><br />$ <span class="token function">git</span> cat-file <span class="token parameter variable">-p</span> 3c201df6a1c4d4c87177e30e93be1df8bfe2fe19<br />tree 62902ec0eca9faceb8fe0a9870b9b6cde75a9545<br />author Abin Simon <span class="token operator"><</span>mail@meain.io<span class="token operator">></span> <span class="token number">1688292123</span> +0530<br />committer Abin Simon <span class="token operator"><</span>mail@meain.io<span class="token operator">></span> <span class="token number">1688292123</span> +0530<br /><br />Initial commit</code></pre>
<blockquote>
<p>You could have also done <code>git cat-file -t refs/heads/master</code></p>
</blockquote>
<p>Well, looks like that is new kind of object. This seems to be a <code>commit</code> object. The contents of the <code>commit</code> object tells us that it contains a <code>tree</code> object with the hash <code>62902ec0eca9faceb8fe0a9870b9b6cde75a9545</code>, which looks like the other object that got added when we did the commit. The <code>commit</code> object also has the information about who the author and committer is, which in this case is both me. Lastly is also shows what the commit message for this commit was.</p>
<p>Now let's look at what the <code>tree</code> object contains.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">git</span> cat-file <span class="token parameter variable">-t</span> 62902ec0eca9faceb8fe0a9870b9b6cde75a9545<br />tree<br /><br />$ <span class="token function">git</span> cat-file <span class="token parameter variable">-p</span> 62901ec0eca9faceb8fe0a9870b9b6cde75a9545<br /><span class="token number">100644</span> blob 4c5b58f323d7b459664b5d3fb9587048bb0296de <span class="token function">file</span></code></pre>
<p>A <code>tree</code> object will contain the state of working directory in the form of other tree and blob objects. In this case, since we just have a single file named <code>file</code>, you will just see a single object. If you see, the file is pointing to the original object that got added when we did a <code>git add file</code>.</p>
<p>Here is what a tree for a more mature repo look like. More <code>tree</code> objects are used inside <code>tree</code> object linked from the <code>commit</code> object to denote folders.</p>
<pre><code>$ git cat-file -p 2e5e84c3ee1f7e4cb3f709ff5ca0ddfc259a8d04
100644 blob 3cf56579491f151d82b384c211cf1971c300fbf8 .dockerignore
100644 blob 02c348c202dd41f90e66cfeb36ebbd928677cff6 .gitattributes
040000 tree ab2ba080c4c3e4f2bc643ae29d5040f85aca2551 .github
100644 blob bdda0724b18c16e69b800e5e887ed2a8a210c936 .gitignore
100644 blob 3a592bc0200af2fd5e3e9d2790038845f3a5cf9b CHANGELOG.md
100644 blob 71a7a8c5aacbcaccf56740ce16a6c5544783d095 CODE_OF_CONDUCT.md
100644 blob f433b1a53f5b830a205fd2df78e2b34974656c7b LICENSE
100644 blob 413072d502db332006536e1af3fad0dce570e727 README.md
100644 blob 1dd7ed99019efd6d872d5f6764115a86b5121ae9 SECURITY.md
040000 tree 918756f1a4e5d648ae273801359c440c951555f9 build
040000 tree 219a6e58af53f2e53b14b710a2dd8cbe9fea15f5 design
040000 tree 5810c119dd4d9a1c033c38c12fae781aeffeafc1 docker
040000 tree f09c5708676cdca6562f10e1f36c9cfd7ee45e07 src
040000 tree e6e1595f412599d0627a9e634007fcb2e32b62e5 website
</code></pre>
<h1 id="making-a-change" tabindex="-1">Making a change <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#making-a-change">#</a></h1>
<p>Let's make a change to the file and see how that works.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token builtin class-name">echo</span> <span class="token string">'blog.meain.io'</span> <span class="token operator">></span> <span class="token function">file</span><br />$ <span class="token function">git</span> commit <span class="token parameter variable">-am</span> <span class="token string">'Use blog link'</span><br /><span class="token punctuation">[</span>master 68ed5aa<span class="token punctuation">]</span> Use blog <span class="token function">link</span><br /> <span class="token number">1</span> <span class="token function">file</span> changed, <span class="token number">1</span> insertion<span class="token punctuation">(</span>+<span class="token punctuation">)</span>, <span class="token number">1</span> deletion<span class="token punctuation">(</span>-<span class="token punctuation">)</span></code></pre>
<p>Here is what it does:</p>
<pre class="language-diff"><code class="language-diff"><span class="token coord">--- commit 2024-07-02 15:33:28.536144046 +0530</span><br /><span class="token coord">+++ update 2023-07-02 15:47:20.841154907 +0530</span><br /><span class="token coord">@@ -17,6 +17,12 @@</span><br /><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">│ │ └── 5b58f323d7b459664b5d3fb9587048bb0296de<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ ├── 62<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ │ └── 901ec0eca9faceb8fe0a9870b9b6cde75a9545<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">│ ├── 67<br /></span><span class="token prefix inserted">+</span><span class="token line">│ │ └── ed5aa2372445cf2249d85573ade1c0cbb312b1<br /></span><span class="token prefix inserted">+</span><span class="token line">│ ├── 8a<br /></span><span class="token prefix inserted">+</span><span class="token line">│ │ └── b377e2f9acd9eaca12e750a7d3cb345065049e<br /></span><span class="token prefix inserted">+</span><span class="token line">│ ├── e5<br /></span><span class="token prefix inserted">+</span><span class="token line">│ │ └── ec63cd761e6ab9d11e7dc2c4c2752d682b36e2<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">│ ├── info<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ └── pack<br /></span><span class="token prefix unchanged"> </span><span class="token line">└── refs</span></span></code></pre>
<p>Well, we added 3 new objects. One of them would be a <code>blob</code> object with the new contents of the file, one would be a <code>tree</code> object and the last one will be a <code>commit</code> object.</p>
<p>Let's trace them again from the <code>HEAD</code> or <code>refs/heads/master</code>.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">git</span> cat-file <span class="token parameter variable">-p</span> refs/heads/master<br />tree 9ab377e2f9acd9eaca12e750a7d3cb345065049e<br />parent 3c201df6a1c4d4c87177e30e93be1df8bfe2fe19<br />author Abin Simon <span class="token operator"><</span>mail@meain.io<span class="token operator">></span> <span class="token number">1688292975</span> +0530<br />committer Abin Simon <span class="token operator"><</span>mail@meain.io<span class="token operator">></span> <span class="token number">1688292975</span> +0530<br /><br />Use blog <span class="token function">link</span><br /><br />$ <span class="token function">git</span> cat-file <span class="token parameter variable">-p</span> 9ab377e2f9acd9eaca12e750a7d3cb345065049e<br /><span class="token number">100644</span> blob e5ec63cd761e6ab9d11e7dc2c4c2752d682b36e2 <span class="token function">file</span><br /><br />$ <span class="token function">git</span> cat-file <span class="token parameter variable">-p</span> e6ec63cd761e6ab9d11e7dc2c4c2752d682b36e2<br />blog.meain.io</code></pre>
<p>Those paying attention might have noticed that the <code>commit</code> object now has an additional key called <code>parent</code> which links to the previous commit as this commit is created on top of the previous commit.</p>
<h1 id="creating-a-branch" tabindex="-1">Creating a branch <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#creating-a-branch">#</a></h1>
<p>About time we created a branch. Let's do that with <code>git branch fix-url</code>.</p>
<pre class="language-diff"><code class="language-diff"><span class="token coord">--- update 2024-07-02 15:47:20.841154907 +0530</span><br /><span class="token coord">+++ branch 2023-07-02 15:55:25.165204941 +0530</span><br /><span class="token coord">@@ -27,5 +28,6 @@</span><br /><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">│ └── pack<br /></span><span class="token prefix unchanged"> </span><span class="token line">└── refs<br /></span><span class="token prefix unchanged"> </span><span class="token line"> ├── heads<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> │ ├── fix-url<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> │ └── master<br /></span><span class="token prefix unchanged"> </span><span class="token line"> └── tags</span></span></code></pre>
<p>This adds a new file under the folder <code>refs/heads</code> with a file as the branch name and the content as the id of the latest commit.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">cat</span> .git/refs/heads/fix-url<br />68ed5aa2372445cf2249d85573ade1c0cbb312b1</code></pre>
<p>This is pretty much all there is to creating a branch. Branches in <code>git</code> are really cheap. Tags also behave the same way, except that they are created under <code>refs/tags</code>.</p>
<p>A file is also added under the <code>logs</code> directory to store the commit history data similar to <code>master</code> branch.</p>
<h1 id="checking-out-a-branch" tabindex="-1">Checking out a branch <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#checking-out-a-branch">#</a></h1>
<p>Checking out in <code>git</code> is git getting the <code>tree</code> object of a commit and updating the files in your worktree to match the state recorded in it. In this case, since we are switching from <code>master</code> to <code>fix-url</code>, both of which point to the same <code>commit</code> and underlying <code>tree</code> object, <code>git</code> does not have anything to do in the working tree.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">git</span> checkout fix-url</code></pre>
<p>The only change that happens when you do a checkout inside <code>.git</code> is that the <code>.git/HEAD</code> file will now point to <code>fix-url</code>.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">cat</span> .git/HEAD<br />ref: refs/heads/fix-url</code></pre>
<p>Wile we are here, let me make a commit. I'm gonna need this to show what merging does later.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token builtin class-name">echo</span> <span class="token string">'https://blog.meain.io'</span><span class="token operator">></span>file<br />$ <span class="token function">git</span> commit <span class="token parameter variable">-am</span> <span class="token string">'Fix url'</span></code></pre>
<h1 id="merging" tabindex="-1">Merging <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#merging">#</a></h1>
<p>There are primarily 3 ways of merging.</p>
<ol>
<li>The simplest and the most easiest is a fast forward merge. In this case you just update the commit a branch is pointing to a commit another branch is pointing to. This pretty much involves copying the hash in <code>refs/heads/fix-url</code> to <code>refs/heads/master</code>.</li>
<li>The second one is rebase merge. In this case we first apply our changes on top of what main is currently pointing to one commit at a time and then perform something similar to a fast forward merge.</li>
<li>The last one would be to just merge two branches using a separate merge commit. This is a bit different in that it will have two <code>parent</code> entries in its commit object. We will go a bit more into this towards the end.</li>
</ol>
<p>First let's see what the graph looks like before a merge.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">git</span> log <span class="token parameter variable">--graph</span> <span class="token parameter variable">--oneline</span> <span class="token parameter variable">--all</span><br />* 42c6318 <span class="token punctuation">(</span>fix-url<span class="token punctuation">)</span> Fix url<br />* 67ed5aa <span class="token punctuation">(</span>HEAD -<span class="token operator">></span> master<span class="token punctuation">)</span> Use blog <span class="token function">link</span><br />* 3c201df Initial commit</code></pre>
<p>Now to perform the merge:</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">git</span> merge fix-url <span class="token comment"># updates refs/heads/master to the hash in refs/heads/fix-url</span></code></pre>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">git</span> log <span class="token parameter variable">--graph</span> <span class="token parameter variable">--oneline</span> <span class="token parameter variable">--all</span><br />* 42c6318 <span class="token punctuation">(</span>HEAD -<span class="token operator">></span> master<span class="token punctuation">)</span> <span class="token punctuation">(</span>fix-url<span class="token punctuation">)</span> Fix url<br />* 67ed5aa Use blog <span class="token function">link</span><br />* 3c201df Initial commit</code></pre>
<h1 id="pushing" tabindex="-1">Pushing <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#pushing">#</a></h1>
<p>Now that we have been playing around with our local <code>git</code> repo for some time, let's see what happen when we push it. What is being sent to the git repo on the other side?</p>
<p>To show this, first let me create another git repo which can be used as remote for this repo.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">mkdir</span> git-talk-2<br />$ <span class="token builtin class-name">cd</span> git-talk-2 <span class="token operator">&&</span> <span class="token function">git</span> init <span class="token parameter variable">--bare</span><br /><br />$ <span class="token builtin class-name">cd</span> <span class="token punctuation">..</span>/git-talk <span class="token operator">&&</span> <span class="token function">git</span> remote <span class="token function">add</span> origin <span class="token punctuation">..</span>/git-talk-2</code></pre>
<p>Btw, this change of adding a new remote is a config change and you can see that change in <code>.git/config</code> file. I'm gonna let you go look what the change was on your own.</p>
<p>Now let's <a href="https://www.youtube.com/watch?v=OuhFTX6yLXQ">push</a>.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">git</span> push origin master</code></pre>
<p>Let's see what changed in our repo.</p>
<pre class="language-diff"><code class="language-diff"><span class="token coord">--- branch 2023-07-02 15:55:25.165204941 +0530</span><br /><span class="token coord">+++ remote 2023-07-02 17:41:05.170923141 +0530</span><br /><span class="token coord">@@ -22,12 +29,18 @@</span><br /><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">│ ├── e5<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ │ └── ec63cd761e6ab9d11e7dc2c4c2752d682b36e2<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ ├── info<br /></span><span class="token prefix unchanged"> </span><span class="token line">│ └── pack<br /></span><span class="token prefix unchanged"> </span><span class="token line">├── ORIG_HEAD<br /></span><span class="token prefix unchanged"> </span><span class="token line">└── refs<br /></span><span class="token prefix unchanged"> </span><span class="token line"> ├── heads<br /></span><span class="token prefix unchanged"> </span><span class="token line"> │ ├── fix-url<br /></span><span class="token prefix unchanged"> </span><span class="token line"> │ └── master<br /></span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> ├── remotes<br /></span><span class="token prefix inserted">+</span><span class="token line"> │ └── origin<br /></span><span class="token prefix inserted">+</span><span class="token line"> │ └── master<br /></span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> └── tags</span></span></code></pre>
<p>It added a new <code>refs/remotes</code> to store the information on what all is available in different remotes.</p>
<p>But what gets sent to the other git repo? It is everything that is in <code>objects</code>, and all the branches and tags under <code>refs</code> that you explicitly push. That is all the other git instance needs to get your entire git history.</p>
<h1 id="references" tabindex="-1">References <a class="direct-link" href="https://blog.meain.io/2023/what-is-in-dot-git/#references">#</a></h1>
<ul>
<li><a href="https://git-scm.com/book/en/v3/Git-Internals-Git-Objects">https://git-scm.com/book/en/v3/Git-Internals-Git-Objects</a></li>
<li><a href="https://matthew-brett.github.io/curious-git/reading_git_objects.html">https://matthew-brett.github.io/curious-git/reading_git_objects.html</a></li>
<li><a href="https://blog.meain.io/2020/bunch-of-git-stuff/">https://blog.meain.io/2020/bunch-of-git-stuff/</a></li>
</ul>
<hr />
<p>Feel free to checkout the discussion at <a href="https://news.ycombinator.com/item?id=37792097">Hacker News</a> and <a href="https://lobste.rs/s/mfo49v/what_is_git_directory">Lobsters</a>.</p>
How I manage my tasks in LogSeq2023-10-17T00:00:00Zhttps://blog.meain.io/2023/logseq-task-management/<p>Here is an overview of how I manage my tasks/todos in <a href="https://logseq.com/">LogSeq</a>. I do use LogSeq to take notes as well, but I'm not gonna go much into that here. I started getting more into LogSeq, like most people, when I got addicted to endlessly watching those non-productive productivity YouTube channels where they switch their note taking app every week and expect to be more productive in the new one just because it has more features.</p>
<blockquote>
<p>Jump to the <a href="https://blog.meain.io/2023/logseq-task-management/#my-logseq-task-management-setup">LogSeq task management setup</a> if you just wanna know how I manage my tasks in LogSeq.</p>
</blockquote>
<h1 id="history" tabindex="-1">History <a class="direct-link" href="https://blog.meain.io/2023/logseq-task-management/#history">#</a></h1>
<p>Before we go into how I manage tasks in LogSeq, let me give you a <s>quick</s> intro how my task/todo management setup has evolved over time. I initially wrote a long blog post explaining all the things that I tried, what I like about each and how it evolved, but I didn't feel like publishing it. This is a <s>much</s> relatively shorter version of it.</p>
<h2 id="google-keep---it-just-works" tabindex="-1">Google Keep - It just works <a class="direct-link" href="https://blog.meain.io/2023/logseq-task-management/#google-keep---it-just-works">#</a></h2>
<p>I started with <a href="https://keep.google.com/">Google Keep</a> back in college. Before this it was all physical notebooks. I think most people have used or at least heard about Keep at some point. For those looking for something simple, it is just bloody awesome if you can look past the fact that Google is in control of your data.</p>
<p>After college, I wanted to switch from Google Keep to something else. For one, I spent most of my time on a desktop and Keep only had a web app. I was also really into writing unnecessary bash scripts that I wanted some way that I can access my tasks from a terminal. These reasons, combined with the fact that Google had access to my super secret todo list, I decided to venture out into the wild west of todo/task management apps.</p>
<p>For those who have not looked in there, it is a nightmare. I was not planning on using anything that I had to create an account for and that ruled out most good looking ones. I also wanted something that I can interact with via bash. There are no shortage of todo apps on the Play store, but I had not found anything that worked well for me. I even tried making one, but only built 3 incomplete ones.</p>
<h2 id="moving-to-taskwarrior" tabindex="-1">Moving to Taskwarrior <a class="direct-link" href="https://blog.meain.io/2023/logseq-task-management/#moving-to-taskwarrior">#</a></h2>
<p>After looking for a while I found out about <a href="https://taskwarrior.org/">taskwarrior</a> via YouTube. It felt like everything that I was looking for and more. It is really powerful if you have not tried it.</p>
<p>Taskwarrior also has integrations with most of the things that I wanted, and if anything was missing, it was just a weekend worth of bash hackery. I had found the perfect (at that time) app for managing my todos. I used it quite extensively for a while, but the mobile app integration was a mess. Sure you can use it via Android and sync using a taskwarrior server, but the app was just ugly and unsuable. I remember having to change the orientation to landscape to edit todos. Also, taking notes on task(had a "plugin") felt like a hack.</p>
<p>Even with these limitations, I did use it for quite some time and was managing my tasks within it. <a href="https://www.youtube.com/watch?v=tijnc65soEI">This</a> is a great talk in case you are interested in checking it out. I had, around the time also tried out <a href="http://todotxt.org/">todo.txt</a>. I did use it for a while, but somehow did not stick to it. There was also <a href="https://play.google.com/store/apps/details?id=io.gitjournal.gitjournal&gl=IT">GitJournal</a> which I tried out for while where it synced through <code>git</code>.</p>
<h2 id="why-i-didn't-use-org-mode" tabindex="-1">Why I didn't use <code>org-mode</code> <a class="direct-link" href="https://blog.meain.io/2023/logseq-task-management/#why-i-didn't-use-org-mode">#</a></h2>
<p>It was around this time that I started to switch to Emacs from Vim/NeoVim. And for anyone who knows about Emacs, I'm sure you also have heard of Org-Mode. The all powerful note taking, task management app. Well, all of that is true, but after all the years of using <code>#</code> as headings in Markdown, it somehow did not felt right using <code>*</code> for headings(probably the silliest argument). Besides, the Android app situation was also messy. People were even suggesting to use termux on Android and to use Emacs inside that. Nah bruh, that ain't it. I think the app situation has gotten slightly better now.</p>
<h1 id="we-are-getting-there" tabindex="-1">We are getting there <a class="direct-link" href="https://blog.meain.io/2023/logseq-task-management/#we-are-getting-there">#</a></h1>
<p>During all of this mess, I organically started evolving a system. It started with me just creating a file in the <code>$HOME</code> directory called <code>things-for-today</code> and copying down things that I have to do for that day from taskwarriror in the morning. Initially, I would recreate the file every morning, but after a while I stopped deleting it and started noting down future todos in it as well instead of taskwarriror. I also started sectioning(a line starting with <code>#</code> followed by something) off items by the date. This evolved into me scheduling items for the near future, again just by creating sections for the day.</p>
<p>After a long time using this, and slowing using taskwarrior less and less, and I created a simple vim plugin (<a href="https://github.com/meain/vim-mtodo">meain/mtodo</a>) to simplify managing tasks and to give it some colors. To use this on the phone, I was just using a simple markdown editor. Here is how it looks in Vim.</p>
<p><img src="https://blog.meain.io/img/vim-mtodo.png" alt="vim-mtodo in use" /></p>
<p>After moving to Emacs, I also ported it over to Emacs(obviously) as a major mode called <a href="https://github.com/meain/dotfiles/blob/master/emacs/.config/emacs/mtodo-mode.el">mtodo-mode.el</a>. Here is how it looks like in my current Emacs setup just for completeness.</p>
<p><img src="https://blog.meain.io/img/emacs-mtodo.png" alt="mtodo-mode.el in use" /></p>
<p>An added advantage of this system is that I can just create another file, in the root, or even in a project root if I wanted to separate out the tasks. I used it as it is for a really long time. I only had one gripe about it in that adding additional notes to a task felt out of place. While I could add notes on tasks below the task, it somehow felt lacking.</p>
<p>I stuck to this for quite a few years. During this time I was managing my notes separately as markdown files which I dumped into a folder heirarchy. My note taking journey is another giant rabbit hole that I am not gonna go into now. I have written a note taking "app" in at least 6 programming languages with some having multiple implementations. I think I had at least 5 different bash based note taking systems.</p>
<h1 id="starting-to-use-logseq" tabindex="-1">Starting to use LogSeq <a class="direct-link" href="https://blog.meain.io/2023/logseq-task-management/#starting-to-use-logseq">#</a></h1>
<p>Last year around this time, I was introduced to <a href="https://logseq.com/">LogSeq</a> during a work meeting. I was initially reluctant(primarily because I am conditioned to hate anything built on top of Electron), but decided to give it a try just for taking notes.</p>
<p>After all the years of building note taking systems, where I had never understood why anyone would need to "link" their notes, I finally understood the power of linking notes. It finally clicked for me and I started using LogSeq more and more for taking notes and generally collecting information.</p>
<p>Just for those who are wondering, I am aware of <a href="https://obsidian.md/">Obsidian</a> and have tried it, but it never felt right for me (I want to like it, but somehow don't), probably something to do with <a href="https://www.youtube.com/watch?v=f3dDVtJ2sec">the kind of note taker that I am</a>. <em>For those who think that was a niche video, you have no idea how deep the rabbit hole goes.</em> <a href="https://www.youtube.com/watch?v=XRpHIa-2XCE">This</a> is a beautiful satirical video around the whole deal.</p>
<h1 id="my-logseq-task-management-setup" tabindex="-1">My LogSeq task management setup <a class="direct-link" href="https://blog.meain.io/2023/logseq-task-management/#my-logseq-task-management-setup">#</a></h1>
<p>Finally getting to the actual thing that this blog is about, how I manage my tasks in LogSeq. I tried a bunch of stuff. I tried what most people recommend, ie using Journal page to write down all my todos and then writing queries to manage them, but here is what finally worked.</p>
<p><strong>I have a setup similar to the <code>mtodo</code> plugin ideas where I just have a list of tasks. I maintain a page for each project/area that I working in the format <code>Backlog/<project></code>. Instead of using date headings, I set scheduled dates. As for completion dates, I make use of <a href="https://github.com/DimitryDushkin/logseq-plugin-task-check-date">DimitryDushkin/logseq-plugin-task-check-date</a>.</strong></p>
<p><img src="https://blog.meain.io/img/logseq-backlog.png" alt="LogSeq task management" /></p>
<p>This worked really well for me. It also took care of the problem that I had not being able to take notes on the todos. Now I can do that and also link to my other notes. The only problem that I was having with this method was that the files where getting bigger and bigger and LogSeq is not good at managing large files. That is when I was reminded of the idea in Org about refililng things. There is no need for me to keep both pending and completed tasks.</p>
<p>I knew you could create plugins for LogSeq, but I was not in the mood to write Javascript. Than again, these are just text files on disk. I can write a simple bash script to move completed tasks over. Well, while you can't really do it easily with simple text manipulation tools, using <a href="https://tree-sitter.github.io/tree-sitter/">tree-sitter</a> to parse out completed items was really simple. With the help of tree-sitter, I was able to whip up something that does it quite easily. You can find the script <a href="https://github.com/meain/dotfiles/blob/21ec02f928f2adbd321e94e4d3e261d2bd8eed7c/scripts/.local/bin/random/%2Clogseq-refile">here</a>.</p>
<p>After using it for a while, I realized that LogSeq graph goes out of sync when I just edit the files from underneath it. A few months later, I gave up and wrote a LogSeq plugin which does it. You can find it at <a href="https://github.com/meain/logseq-plugin-refile">meain/logseq-plugin-refile</a>. Now that I started writing plugins for LogSeq, I also wrote <a href="https://github.com/meain/logseq-plugin-favorite-jump">meain/logseq-plugin-favorite-jump</a> so that I can easily open these backlog files. While I was at it, I also wrote something simple which will play a small sound when you mark something as done. You can find that one at <a href="https://github.com/meain/logseq-plugin-noisy">meain/logseq-plugin-noisy</a>.</p>
<p>All of this combined with a few journal page queries to view today's and missed tasks is my entire task management system(for now). I replace the default journal queries with the following ones (you can add these to <code>config.edn</code>).</p>
<pre class="language-clojure"><code class="language-clojure"> <span class="token symbol">:feature/disable-scheduled-and-deadline-query?</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /> <span class="token symbol">:default-queries</span><br /> <span class="token punctuation">{</span><span class="token symbol">:journals</span><br /> <span class="token punctuation">[</span><span class="token punctuation">{</span><span class="token symbol">:breadcrumb-show?</span> <span class="token boolean">false</span><span class="token punctuation">,</span><br /> <span class="token symbol">:collapsed?</span> <span class="token boolean">false</span><span class="token punctuation">,</span><br /> <span class="token symbol">:group-by-page?</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /> <span class="token symbol">:inputs</span> <span class="token punctuation">[</span><span class="token symbol">:today</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token symbol">:query</span> <span class="token punctuation">[</span><span class="token symbol">:find</span> <span class="token punctuation">(</span><span class="token function">pull</span> ?b <span class="token punctuation">[</span>*<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token symbol">:in</span> $ ?duetoday <span class="token symbol">:where</span> <span class="token punctuation">[</span>?b <span class="token symbol">:block/marker</span> ?m<span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token keyword">contains?</span> <span class="token operator">#</span><span class="token punctuation">{</span><span class="token string">"LATER"</span> <span class="token string">"TODO"</span> <span class="token string">"DOING"</span> <span class="token string">"NOW"</span><span class="token punctuation">}</span> ?m<span class="token punctuation">)</span><span class="token punctuation">]</span><br /> <span class="token punctuation">[</span>?b <span class="token symbol">:block/scheduled</span> ?d<span class="token punctuation">]</span> <span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token keyword">=</span> ?d ?duetoday<span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token symbol">:result-transform</span><br /> <span class="token punctuation">(</span><span class="token keyword">fn</span> <span class="token punctuation">[</span>result<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">sort-by</span> <span class="token punctuation">(</span><span class="token keyword">fn</span> <span class="token punctuation">[</span>h<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token function">get-in</span> h <span class="token punctuation">[</span><span class="token symbol">:block/scheduled</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">)</span> result<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token symbol">:title</span> <span class="token string">"📤 SCHEDULED"</span><span class="token punctuation">}</span><br /> <span class="token punctuation">{</span><span class="token symbol">:breadcrumb-show?</span> <span class="token boolean">false</span><span class="token punctuation">,</span><br /> <span class="token symbol">:collapsed?</span> <span class="token boolean">false</span><span class="token punctuation">,</span><br /> <span class="token symbol">:group-by-page?</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /> <span class="token symbol">:inputs</span> <span class="token punctuation">[</span><span class="token symbol">:today</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token symbol">:query</span> <span class="token punctuation">[</span><span class="token symbol">:find</span> <span class="token punctuation">(</span><span class="token function">pull</span> ?b <span class="token punctuation">[</span>*<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token symbol">:in</span> $ ?duetoday <span class="token symbol">:where</span> <span class="token punctuation">[</span>?b <span class="token symbol">:block/marker</span> ?m<span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token keyword">contains?</span> <span class="token operator">#</span><span class="token punctuation">{</span><span class="token string">"LATER"</span> <span class="token string">"TODO"</span><span class="token punctuation">}</span> ?m<span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token punctuation">[</span>?b <span class="token symbol">:block/scheduled</span> ?d<span class="token punctuation">]</span><br /> <span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token keyword"><</span> ?d ?duetoday<span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token symbol">:result-transform</span><br /> <span class="token punctuation">(</span><span class="token keyword">fn</span> <span class="token punctuation">[</span>result<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token keyword">sort-by</span> <span class="token punctuation">(</span><span class="token keyword">fn</span> <span class="token punctuation">[</span>h<span class="token punctuation">]</span> <span class="token punctuation">(</span><span class="token function">get-in</span> h <span class="token punctuation">[</span><span class="token symbol">:block/scheduled</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">)</span> result<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token symbol">:title</span> <span class="token string">"🏃 MISSED"</span><span class="token punctuation">}</span><span class="token punctuation">]</span><span class="token punctuation">}</span><span class="token punctuation">,</span></code></pre>
<p>Well, that's about it. I don't know how this will change in the future, but I am really liking the setup so far 🤞🏼.</p>
Navigating around in your shell2023-11-16T00:00:00Zhttps://blog.meain.io/2023/navigating-around-in-shell/<p>I have been using terminals for a long time, initially because I thought they looked cool, and later because I genuinely found them to be easier/faster to get stuff done. And since I've been at it for a while, navigating through directories is something I think I've gotten good at. In this blog, I would like to give some tips on ways you can navigate around in your shell quickly.</p>
<p><em>I rarely type <code>cd</code> to change directories. I get the same feeling you get when you see people google for google on google when I see people typing <code>cd ..</code> repeatedly.</em></p>
<h1 id="basics" tabindex="-1">Basics <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#basics">#</a></h1>
<h2 id="using-cd" tabindex="-1">Using <code>cd</code> <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#using-cd">#</a></h2>
<p>We all know the basics, ie <code>cd <dir></code> to navigate to <code>dir</code>.
Here a few other things ways you can use <code>cd</code>:</p>
<ul>
<li><code>cd</code> (no args) is equivalent to <code>cd ~</code></li>
<li><code>cd -</code> takes you to the previous(not parent) directory</li>
<li><code>cd ..</code> takes you to the parent directory</li>
</ul>
<h2 id="make-use-of-that-tab-key" tabindex="-1">Make use of that <code>tab</code> key <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#make-use-of-that-tab-key">#</a></h2>
<p>This might sound simple, but you have no idea how many times I've had the painful experience of watching people type out the path by had and even end typing them incorrectly.</p>
<h2 id="let's-add-some-useful-aliases" tabindex="-1">Let's add some useful aliases <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#let's-add-some-useful-aliases">#</a></h2>
<p>Now that we know some basics, lets add some aliases.</p>
<p>Here are the ones I really like. I would suggest adding aliases for going to previous dir, and multiple levels of parent dirs. I initial got this idea from <a href="https://github.com/ohmyzsh/ohmyzsh">ohmyzsh/ohmyzsh</a>. You can find these in my <a href="https://github.com/meain/dotfiles/blob/f7ac724386f9dfb24eda53191146795b788b0f06/zsh/.config/zsh/aliases#L4-L9"><code>dotfiles</code></a>.</p>
<pre class="language-bash"><code class="language-bash"><span class="token builtin class-name">alias</span> -- -<span class="token operator">=</span><span class="token string">'cd -'</span><br /><span class="token builtin class-name">alias</span> <span class="token punctuation">..</span><span class="token operator">=</span><span class="token string">'cd ..'</span><br /><span class="token builtin class-name">alias</span> <span class="token punctuation">..</span>.<span class="token operator">=</span><span class="token string">'cd ../..'</span><br /><span class="token builtin class-name">alias</span> <span class="token punctuation">..</span><span class="token punctuation">..</span><span class="token operator">=</span><span class="token string">'cd ../../..'</span><br /><span class="token builtin class-name">alias</span> <span class="token punctuation">..</span><span class="token punctuation">..</span>.<span class="token operator">=</span><span class="token string">'cd ../../../..'</span><br /><span class="token builtin class-name">alias</span> <span class="token punctuation">..</span><span class="token punctuation">..</span><span class="token punctuation">..</span><span class="token operator">=</span><span class="token string">'cd ../../../../..'</span></code></pre>
<h2 id="make-use-of-globs" tabindex="-1">Make use of globs <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#make-use-of-globs">#</a></h2>
<p>If you know only parts of a path, you can always use globs and let shell figure out the exact path. Use this along with tab completion to make it even more powerful.</p>
<p>For example you can do <code>cd src/**/color</code> to find a directory named <code>color</code> in the current working directory.</p>
<h1 id="builtins" tabindex="-1">Builtins <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#builtins">#</a></h1>
<h2 id="auto-fix-small-errors" tabindex="-1">Auto fix small errors <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#auto-fix-small-errors">#</a></h2>
<p>Since my blog is mostly read by humans(I'm sorry bots, you are in a minority), and since humans make mistakes, this one is for you. I would suggest enabling these two options. The first one ignores the case of the argument and the second one auto fixes any small spelling mistakes.</p>
<pre class="language-bash"><code class="language-bash"><span class="token comment"># for bash</span><br /><span class="token builtin class-name">shopt</span> nocaseglob <span class="token comment"># ignore case when matching</span><br /><span class="token builtin class-name">shopt</span> cdspell <span class="token comment"># fix common spelling mistakes</span><br /><br /><span class="token comment"># for zsh</span><br />setopt nocaseglob <span class="token comment"># ignore case</span><br />setopt correct <span class="token comment"># correct spelling mistakes</span></code></pre>
<p><em>You might be interested in <a href="https://ss64.com/bash/shopt.html">shopt</a> and <a href="https://zsh.sourceforge.io/Doc/Release/Options.html">setopt</a> manuals.</em></p>
<h2 id="for-the-lazy-ones" tabindex="-1">For the lazy ones <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#for-the-lazy-ones">#</a></h2>
<p>Why type <code>cd</code> if you don't have to. If you enable to following option, you can just type the name of the folder and if there is no binary by that name in your <code>$PATH</code>, your shell will <code>cd</code> into it that directory.</p>
<pre class="language-bash"><code class="language-bash"><span class="token builtin class-name">shopt</span> <span class="token parameter variable">-s</span> autocd <span class="token comment"># for bash</span><br />setopt auto_cd <span class="token comment"># for zsh</span></code></pre>
<h2 id="remembering-a-directory" tabindex="-1">Remembering a directory <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#remembering-a-directory">#</a></h2>
<p>What if you wanna move around a lot, but wanIf you're frequently on the move but need to stay organized and revisit specific directories easily, utilizing pushd and popd can be incredibly beneficial.t to keep track of certain directories specifically to come back to. <code>pushd</code> and <code>popd</code> are you friends.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">pushd</span> dir1 <span class="token comment"># in dir1</span><br /><span class="token builtin class-name">cd</span> dir2 <span class="token comment"># in dir2</span><br /><span class="token function">pushd</span> dir3 <span class="token comment"># in dir3</span><br /><span class="token builtin class-name">cd</span> dir4<br /><span class="token builtin class-name">cd</span> dir5<br /><span class="token function">popd</span> <span class="token comment"># in dir3</span><br /><span class="token function">popd</span> <span class="token comment"># in dir1</span></code></pre>
<h2 id="bookmarks" tabindex="-1">Bookmarks <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#bookmarks">#</a></h2>
<p><a href="https://til.hashrocket.com/posts/xsavbhlrz4-shortcuts-with-hash-d-in-zsh"><code>hash</code></a> is a way for you to "bookmark" directories. I use it to "bookmark" directories that I visit often. By default, the bookmarks are only per session, but you can add a bit more code to make it persistent.</p>
<p>The best part is that you can use these as prefixes to construct the full paths. For example if I have bookmark <code>~/.local/share</code> as <code>~share</code>, I can later do things like <code>cd ~share/emacs</code>.</p>
<p>Here are the additions that I have:</p>
<pre class="language-bash"><code class="language-bash"><span class="token punctuation">[</span> <span class="token parameter variable">-f</span> <span class="token string">"<span class="token environment constant">$HOME</span>/.local/share/.zsh_dir_hashes"</span> <span class="token punctuation">]</span> <span class="token operator">&&</span> <span class="token string">"<span class="token environment constant">$HOME</span>/.local/share/.zsh_dir_hashes"</span><br /><span class="token function-name function">hashdir</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token punctuation">[</span> <span class="token parameter variable">-z</span> <span class="token string">"<span class="token variable">$1</span>"</span> <span class="token punctuation">]</span> <span class="token operator">&&</span> <span class="token builtin class-name">echo</span> <span class="token string">"Pass hash to use for dir"</span> <span class="token operator">&&</span> <span class="token builtin class-name">return</span> <span class="token number">1</span><br /> <span class="token builtin class-name">hash</span> <span class="token parameter variable">-d</span> <span class="token string">"<span class="token variable">$1</span>"</span><span class="token operator">=</span><span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token builtin class-name">pwd</span><span class="token variable">)</span></span>"</span><br /> <span class="token builtin class-name">echo</span> <span class="token builtin class-name">hash</span> <span class="token parameter variable">-d</span> <span class="token string">"<span class="token variable">$1</span>"</span><span class="token operator">=</span><span class="token punctuation">\</span>"<span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token builtin class-name">pwd</span><span class="token variable">)</span></span>"</span><span class="token punctuation">\</span>" <span class="token operator">>></span><span class="token string">"<span class="token environment constant">$HOME</span>/.local/share/.zsh_dir_hashes"</span><br /><span class="token punctuation">}</span></code></pre>
<h3 id="use-cdpath-env-variable" tabindex="-1">Use <code>CDPATH</code> env variable <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#use-cdpath-env-variable">#</a></h3>
<p>You can set this env variable with the paths that you would like to always cd into.</p>
<p>For instance, if I set it as <code>export CDPATH=$HOME/.config:$HOME/.local/share</code>, I can cd into any dirs in <code>$HOME/.config</code> and <code>$HOME/.local/share</code> form any location. Moreover, these paths will be incorporated into your shell completions.</p>
<h1 id="scripts-and-aliases" tabindex="-1">Scripts and aliases <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#scripts-and-aliases">#</a></h1>
<h2 id="make-a-directory-and-cd-into-it" tabindex="-1">Make a directory and cd into it <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#make-a-directory-and-cd-into-it">#</a></h2>
<p>I more of than not find myself having to create a directory and cd into it. I use an function called <code>take</code> for this. This is another one of those gems I stole from <code>oh-my-zsh</code>.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function-name function">take</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">mkdir</span> <span class="token parameter variable">-p</span> <span class="token string">"<span class="token variable">$1</span>"</span> <span class="token operator">&&</span> <span class="token builtin class-name">cd</span> <span class="token string">"<span class="token variable">$1</span>"</span><br /><span class="token punctuation">}</span></code></pre>
<h2 id="navigating-to-project-root" tabindex="-1">Navigating to project root <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#navigating-to-project-root">#</a></h2>
<p>Another common usecase is to go the root of the project. Let's also simplify it by using <code>git</code> to figure out the root of the project.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function-name function">r</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token builtin class-name">cd</span> <span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token function">git</span> rev-parse --show-toplevel <span class="token operator"><span class="token file-descriptor important">2</span>></span>/dev/null<span class="token variable">)</span></span>"</span><br /><span class="token punctuation">}</span></code></pre>
<h2 id="navigate-to-a-project-under-this-dir" tabindex="-1">Navigate to a project under this dir <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#navigate-to-a-project-under-this-dir">#</a></h2>
<p>Like most people, I too maintain a dir under which I keep all my project. This function lets me pick one of those projects (using <code>fzf</code> to filter). It looks for all dirs with a <code>.git</code> in them and find their parent dirs.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function-name function">jj</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token builtin class-name">cd</span> <span class="token string">"<span class="token variable">${1<span class="token operator">:-</span>.}</span>/<span class="token variable"><span class="token variable">$(</span><span class="token function">find</span> <span class="token builtin class-name">.</span> <span class="token parameter variable">-maxdepth</span> <span class="token number">5</span> <span class="token parameter variable">-type</span> d <span class="token parameter variable">-name</span> .git <span class="token operator">|</span> <span class="token function">sed</span> <span class="token string">'s|/.git$||'</span> <span class="token operator">|</span> fzf <span class="token parameter variable">--preview</span> <span class="token string">'tree -L 2 ./{}'</span><span class="token variable">)</span></span>"</span><br /><span class="token punctuation">}</span></code></pre>
<h2 id="quickly-create-a-tmp-dir" tabindex="-1">Quickly create a tmp dir <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#quickly-create-a-tmp-dir">#</a></h2>
<p>Whenever I want to test out something, or look at(clone) a project just temporarily, I used to create a dir under <code>/tmp</code> and then do it there. This script automates that. The actual script that I use also does a <code>git init</code> so that I can track and diff files.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function-name function">tmp</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token punctuation">[</span> <span class="token string">"<span class="token variable">$1</span>"</span> <span class="token operator">=</span> <span class="token string">"view"</span> <span class="token punctuation">]</span> <span class="token operator">&&</span> <span class="token builtin class-name">cd</span> /tmp/workspaces <span class="token operator">&&</span> <span class="token builtin class-name">cd</span> <span class="token variable"><span class="token variable">$(</span><span class="token function">ls</span> <span class="token parameter variable">-t</span> <span class="token operator">|</span> fzf <span class="token parameter variable">--preview</span> <span class="token string">'ls -A {}'</span><span class="token variable">)</span></span> <span class="token operator">&&</span> <span class="token builtin class-name">return</span> <span class="token number">0</span><br /> <span class="token assign-left variable">r</span><span class="token operator">=</span><span class="token string">"/tmp/workspaces/<span class="token variable"><span class="token variable">$(</span>xxd <span class="token parameter variable">-l3</span> <span class="token parameter variable">-ps</span> /dev/urandom<span class="token variable">)</span></span>"</span><br /> <span class="token function">mkdir</span> <span class="token parameter variable">-p</span> <span class="token parameter variable">-p</span> <span class="token string">"<span class="token variable">$r</span>"</span> <span class="token operator">&&</span> <span class="token function">pushd</span> <span class="token string">"<span class="token variable">$r</span>"</span><br /><span class="token punctuation">}</span></code></pre>
<h2 id="one-off-dirs" tabindex="-1">One-off dirs <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#one-off-dirs">#</a></h2>
<p>One more. I get a lot of data being sent to be as zips, and when I want to look at them, just like before, I create a dir in <code>tmp</code>, unzip it into that dir and look into the logs. After a while I automated these steps. This can sound like a me problem, but just wanted to show you the kind of things you can do.</p>
<pre class="language-bash"><code class="language-bash">,<span class="token function-name function">showme</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token punctuation">[</span> <span class="token parameter variable">-z</span> <span class="token string">"<span class="token variable">$1</span>"</span> <span class="token punctuation">]</span> <span class="token operator">&&</span> <span class="token builtin class-name">echo</span> <span class="token string">"Pass file to show"</span> <span class="token operator">&&</span> <span class="token builtin class-name">return</span> <span class="token number">1</span><br /> <span class="token assign-left variable">tmp</span><span class="token operator">=</span><span class="token string">"<span class="token variable"><span class="token variable">$(</span>mktemp <span class="token parameter variable">-d</span><span class="token variable">)</span></span>"</span><br /> <span class="token function">unzip</span> <span class="token string">"<span class="token variable">$1</span>"</span> <span class="token parameter variable">-d</span> <span class="token string">"<span class="token variable">$tmp</span>"</span><br /> <span class="token builtin class-name">cd</span> <span class="token string">"<span class="token variable">$tmp</span>"</span> <span class="token operator">||</span> <span class="token builtin class-name">return</span> <span class="token number">1</span><br /> <span class="token function">clear</span> <span class="token operator">&&</span> <span class="token builtin class-name">echo</span> <span class="token operator">&&</span> <span class="token builtin class-name">echo</span> <span class="token string">"<span class="token variable">$1</span>"</span> <span class="token operator">&&</span> lsd<br /><span class="token punctuation">}</span></code></pre>
<h1 id="external-tools" tabindex="-1">External tools <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#external-tools">#</a></h1>
<h2 id="fzf" tabindex="-1">fzf <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#fzf">#</a></h2>
<p><a href="https://github.com/junegunn/fzf"><code>fzf</code></a> is a super <a href="https://github.com/search?q=repo%3Ameain%2Fdotfiles+fzf&type=code">versatile</a> tool. I use it a lot in scripts, but by default it adds a <code>ctrl+t</code> shortcut to select files and folders.</p>
<h2 id="intelligent-cd" tabindex="-1">Intelligent <code>cd</code> <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#intelligent-cd">#</a></h2>
<p>There's a multitude of tools available that learn the directories you frequently access and allow you to swiftly "jump" into them by typing partial names. I personally rely on rupa/z due to my familiarity with it, but there are several other options worth exploring, such as wting/autojump, ajeetdsouza/zoxide (the Rust one for extra credibility), and gsamokovarov/jump. Each offers its own set of features and advantages, catering to different preferences and workflows.</p>
<h2 id="using-a-tui-file-browser" tabindex="-1">Using a TUI file browser <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#using-a-tui-file-browser">#</a></h2>
<p>I grew up using <a href="https://github.com/ranger/ranger">ranger/ranger</a> but switched over to <a href="https://github.com/gokcehan/lf">gokcehan/lf</a> at some point and I use it as way to select the dir that I wanna jump into. The following code is taken straight from <code>lf</code>'s wiki. With this, you can type <code>lfcd</code>, then go to the dir you want by navigating a TUI browser and then exit to land on that dir.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function-name function">lfcd</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token assign-left variable">dir</span><span class="token operator">=</span><span class="token variable"><span class="token variable">$(</span>lf -print-last-dir <span class="token string">"<span class="token variable">$@</span>"</span><span class="token variable">)</span></span><br /> <span class="token keyword">while</span> <span class="token operator">!</span> <span class="token builtin class-name">cd</span> <span class="token string">"<span class="token variable">$dir</span>"</span> <span class="token operator"><span class="token file-descriptor important">2</span>></span> /dev/null<br /> <span class="token keyword">do</span><br /> <span class="token assign-left variable">dir</span><span class="token operator">=</span><span class="token variable"><span class="token variable">$(</span><span class="token function">dirname</span> <span class="token string">"<span class="token variable">$dir</span>"</span><span class="token variable">)</span></span><br /> <span class="token keyword">done</span><br /><span class="token punctuation">}</span></code></pre>
<h2 id="fuzzy-tui-navigation" tabindex="-1">Fuzzy tui navigation <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#fuzzy-tui-navigation">#</a></h2>
<p>Another option is to use something like <a href="https://github.com/Canop/broot">Canop/broot</a> to select a dir. I don't use this much, but have found this useful in the past. You can find how to do that <a href="https://github.com/Canop/broot#find-a-directory-then-cd-to-it">here</a>.</p>
<h1 id="bonus" tabindex="-1">Bonus <a class="direct-link" href="https://blog.meain.io/2023/navigating-around-in-shell/#bonus">#</a></h1>
<p>I always wondered if <code>cd .</code> had a purpose. I found this out from someone on Mastodon a while ago (unfortunately, I don't remember who).</p>
<p>If you delete a folder from a different terminal session and subsequently recreate a new folder with the same name, using cd . allows you to navigate into the newly created folder without changing your current directory. This command essentially refreshes the current directory, enabling access to the newly created directory with the same name.</p>
Just enough CORS to not get stuck2024-02-08T00:00:00Zhttps://blog.meain.io/2024/just-enough-cors/<p>Cross-Origin Resource Sharing or CORS is one of those things that I have had to explain to a lot of junior folks. I feel like it is partly because there is a lot of unhelpful and confusing information around it. In this blog, I hope to give a high level overview of what the idea behind CORS is, why it is required and most importantly how to avoid getting CORS errors in your browser.</p>
<p><img src="https://blog.meain.io/img/CORS.png" alt="CORS comic" /></p>
<p><em>Image credit: <a href="https://www.goodtechthings.com/">Good Tech Things</a></em></p>
<p>For most people seeing the below error is enough to make them give up on the project or to start searching for "how to disable CORS in my browser".</p>
<p><img src="https://blog.meain.io/img/cors-error.png" alt="Screenshot of CORS error in Firefox" /></p>
<h1 id="why-do-we-need-cors" tabindex="-1">Why do we need CORS <a class="direct-link" href="https://blog.meain.io/2024/just-enough-cors/#why-do-we-need-cors">#</a></h1>
<p>Before we get into what CORS is, let's see why it is needed in the first place. As always, the reason why we need more complex things is because of security.</p>
<p>CORS is necessary when you want your JavaScript code running in the browser to be talking to a different domain. For example, you might have your UI on <code>ui.example.com</code> and your backend on <code>backend.example.com</code>. Now, if you want your JavaScript on the <code>ui.example.com</code> page to be able to make a <code>POST</code> request to your backend at <code>backend.example.com</code>, then the backend server should allow for it to happen via CORS.</p>
<p><strong>Why is this a problem. Can't we just allow requests to any domain?</strong></p>
<p>Good question. Let me paint you a picture. Let's say you have your bank account with ABC Bank. You wake up in the <s>morning</s> afternoon and start your day off by checking your bank balance. I would ask why you are checking your bank balance first thing in the morning, but I digress. Let's see how your bank website might go about letting you do this.</p>
<p>You log onto <code>abcbank.com</code> with your username and password. It loads and HTML page. Now you click "See balance" button. The website makes a request to <code>api.abcbank.com</code> and get the balance. It shows your balance as 500. You instantly donate 80 to FSF(or EFF if you think FSF is too edgy) so that your balance is 420. To perform these operations(getting balance and doing a transfer), it uses cookies that you have generated when you logged in to authenticate you.</p>
<p>Now that you are done with your morning banking stuff, you close the tab and go check your email. You see a message about Raspberry Pi finally being available. You instantly click to open the message. Suddenly you get a notification on you phone saying you just transferred 351 to a different account.</p>
<p>What happened there? Looks like email about Raspberry Pi being available was a scam and the link you clicked ended up making a request to your bank to do transfer the money. You browser happily sends the auth token along with the request even though the request was made from <code>rpiforsale.com</code>. This particular kind of vulnerability is known as Cross-Site Request Forgery (CSRF).</p>
<p>Suddenly you wake up from the dream and realize you live in a world where browsers are not that dumb. All your money is safe to spend of useless things of your choice.</p>
<h1 id="woah%2C-i-need-cors.-what-is-it%3F" tabindex="-1">Woah, I need CORS. What is it? <a class="direct-link" href="https://blog.meain.io/2024/just-enough-cors/#woah%2C-i-need-cors.-what-is-it%3F">#</a></h1>
<p>I'm glad you came around. Now that we have seen why you need it, lets see what it is.</p>
<p>By default, your browser does not allow you to make requests to a different domain. This is known as the <a href="https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy">same-origin policy</a>.
CORS at its core is a way for a sever to let the browser know what kind of requests it is allowed to make and what all data is allowed to be sent in the request. For example it can say, "I allow <code>GET</code> requests from <code>example.com</code> and <code>POST</code> requests from <code>example.com</code> with <code>Content-Type</code> as <code>application/json</code>".</p>
<p><img src="https://blog.meain.io/img/cors_principle.png" alt="MDN Cors image" /></p>
<p><em>Here is an image I took from <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS">MDN docs</a> explaining CORS. Something visual to look at after staring at a wall of text.</em></p>
<h1 id="how-does-it-work%3F" tabindex="-1">How does it work? <a class="direct-link" href="https://blog.meain.io/2024/just-enough-cors/#how-does-it-work%3F">#</a></h1>
<p>Good question. Let see how does the whole thing actually works. Every time your browser has to make any request that can cause side-effects(HTTP methods other than <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/GET"><code>GET</code></a>, or <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST"><code>POST</code></a> with certain <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types">MIME types</a>), your browser does something called a "preflight" check. This is way for the browser to ask the server what it is allowed to send the request that it is about to.</p>
<p>It does this by sending an <code>OPTIONS</code> request with headers like <code>Access-Control-Request-Method</code>,<code>Access-Control-Request-Headers</code>,<code>Origin</code>,<code>Host</code>,<code>User-Agent</code> and your server can take a look at those and decide if it want to allow the upcoming request. Response headers like <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials"><code>Access-Control-Allow-Credentials</code></a> can let the browser know whether or not send credentials in the request.</p>
<p><em>For those who don't know, an <code>OPTIONS</code> request is just another method like <code>GET</code> or <code>POST</code> that the server can handle. You can process it just like working on any other request.</em></p>
<p>Let's see an example of how this would play out our in our original bank scenario. When the RPI scam website asks the browser to make a request to the bank to do the transfer, the browser will do a "preflight" check. It does this by making an <code>OPTIONS</code> request to the bank's api server(<a href="http://api.abcbank.com/">api.abcbank.com</a>). The important bit here is that the browser would send the <code>Origin</code> header as <code>rpiforsale.com</code> in the request. Then bank's backend, on seeing that the request does not originate from the bank's website returns a <code>403</code> status code. When the browser gets this, it fails the POST request and you money will remain safe with you 😮💨.</p>
<h1 id="non-solutions" tabindex="-1">Non solutions <a class="direct-link" href="https://blog.meain.io/2024/just-enough-cors/#non-solutions">#</a></h1>
<h3 id="accept-everything-from-everywhere" tabindex="-1">Accept everything from everywhere <a class="direct-link" href="https://blog.meain.io/2024/just-enough-cors/#accept-everything-from-everywhere">#</a></h3>
<p>This is useful when you are just getting started or don't care who/what is making the request. This is the same as saying "I don't care who you are, you can come in and take whatever you want". At the minimum, should only be allowing requests only from the domains that you are expecting requests from. The problem is that a lot of "tutorials" just ask you to use something like <a href="https://github.com/corydolphin/flask-cors">corydolphin/flask-cors</a> which basically does the same thing. This is also what shows up at the top when you search for "python cors" which is not helpful. The <a href="https://stackoverflow.com/questions/25594893/how-to-enable-cors-in-flask">StackOverflow</a> top answers also points you in the same direction.</p>
<h3 id="using-browser-plugins-to-avoid-cors" tabindex="-1">Using browser plugins to avoid CORS <a class="direct-link" href="https://blog.meain.io/2024/just-enough-cors/#using-browser-plugins-to-avoid-cors">#</a></h3>
<p>I have seen a couple of junior folks saying "you have to install the CORS plugin to fix that issue". They are referring to plugins like <a href="https://chromewebstore.google.com/detail/allow-cors-access-control/lhobafahddgcelffkeicbaginigeejlf">this</a> which "disable" preflight checks. This, first of all is bad, and second of all will only work on browsers where you have to installed and is not something you should be asking other people to do.</p>
<h3 id="seating-mode-as-no-cors" tabindex="-1">Seating <code>mode</code> as <code>no-cors</code> <a class="direct-link" href="https://blog.meain.io/2024/just-enough-cors/#seating-mode-as-no-cors">#</a></h3>
<p>Another place I've seen people get stuck is folks seeing an option to set <code>mode</code> as <code>no-cors</code>. This, although avoids preflight checks has a couple of issues. You cannot use credentials plus you cannot read the response body.</p>
<h1 id="what-to-actually-do" tabindex="-1">What to actually do <a class="direct-link" href="https://blog.meain.io/2024/just-enough-cors/#what-to-actually-do">#</a></h1>
<p>Now that we have discussed what not to do, let's see what you should be doing.</p>
<p>You should treat <code>OPTIONS</code> as just another method on the endpoint and properly handle it. "Properly handling it" at the bare minimum includes making sure that the origin specified there is something you are expecting a request from. You can also leverage the data in the other headers that the browser sends you, ie method, headers etc to determine if you should be accepting the upcoming request. If you don't think the request should come through, then you can return a 403 back to indicate that to the browser.</p>