pages tagged linuxwshhttps://rtime.felk.cvut.cz/~sojka/tags/linux/wshikiwiki2023-03-31T22:37:11Zgit-sarah: Edit files locally, compile and test remotelyhttps://rtime.felk.cvut.cz/~sojka/blog/git-sarah/2023-03-31T22:37:11Z2019-05-10T22:04:24Z
<p>
Recently, I worked on a <a href="https://github.com/CTU-IIG/kcf">visual object tracker implemented in C++</a>. The
project has several compile configurations and a time-consuming test
suite, some parts of which need a GPU not present in my laptop.
Because many parts of the code depend heavily on compile
configuration, changing them requires recompilation and ideally also
retesting of all configurations to make sure that everything works
correctly. Since my development laptop not only lacks the GPU needed
for testing but also has only four cores, which makes the compilation
unnecessarily slow, I looked for an easy way to compile and test the
code being edited on a powerful remote server.
</p>
<p>
Initially, I tried to use <code>git commit</code> and <code>git push</code> to capture the
state of the working directory and transfer it to the remote server,
but this was not ideal for many reasons. Then I discovered <code>git stash
create</code>, which captures the state of the working directory and creates
a “volatile” commit object not attached to any branch. This command
became the basis of my <code>git-sarah</code> script, which is an acronym
standing for <i>git stash and run at host</i>, and you can see find its
code below.
</p>
<p>
With <code>git-sarah</code> script, instead of instructing my editor or IDE to
run:
</p>
<div class="org-src-container">
<pre class="src src-sh">make
</pre>
</div>
<p>
I instruct it to run the following:
</p>
<div class="org-src-container">
<pre class="src src-sh">git sarah example.org dir -- make
</pre>
</div>
<p>
This command copies the current state of the work tree to <code>$HOME/dir</code>
on the <i>example.org</i> server and runs <code>make</code> in the directory
corresponding to the current directory on the local host. For example,
if the current local directory is <code>~/projects/kcf/src</code>, with <code>kcf</code>
being the root directory of the git repository, the <code>make</code> command
will be run in <code>$HOME/dir/src</code> on the server. In the output (both
stdout and stderr) of the command, the path of the server directory,
e.g., <code>/home/mylogin/dir</code> is replaced with its local counterpart:
<code>/home/me/projects/kcf</code>.
</p>
<p>
If a single command (<code>make</code>) is not enough, you can instruct
<code>git-sarah</code> to run commands from a script (e.g., <code>commands.sh</code>), which
need not to be stored in your git repository. For that, run:
</p>
<div class="org-src-container">
<pre class="src src-sh">git sarah example.org dir -- sh < ./commands.sh
</pre>
</div>
<p>
The code of <code>git-sarah</code> (further development happens on <a href="https://github.com/wentasah/git-sarah">GitHub</a>) is quite simple:
</p>
<div class="org-src-container">
<pre class="src src-sh"><span class="org-comment-delimiter">#</span><span class="org-comment">!/bin/</span><span class="org-keyword">bash</span>
<span class="org-builtin">set</span> -e -o pipefail
<span class="org-variable-name">OPTS_SPEC</span>=<span class="org-string">"\</span>
<span class="org-string">git sarah <host> <remote_dir> [ -- ] [ command ... ]</span>
<span class="org-string">git stash and run at host</span>
<span class="org-string">--</span>
<span class="org-string">h,help show the help</span>
<span class="org-string">"</span>
<span class="org-builtin">eval</span> <span class="org-string">"$(</span><span class="org-sh-quoted-exec">echo "$OPTS_SPEC" | git rev-parse --parseopt -- "$@" || echo exit $?</span><span class="org-string">)"</span>
<span class="org-builtin">echo</span> <span class="org-string">"$*"</span>
<span class="org-keyword">while</span> <span class="org-string">\[</span>[ $<span class="org-variable-name">#</span> -gt 0 ]]; <span class="org-keyword">do</span>
<span class="org-keyword">case</span> <span class="org-string">"$1"</span><span class="org-keyword"> in</span>
-h) <span class="org-keyword">exit</span> 0;;
--) <span class="org-builtin">shift</span>; <span class="org-keyword">break</span>;;
<span class="org-keyword">esac</span>
<span class="org-builtin">shift</span>
<span class="org-keyword">done</span>
<span class="org-variable-name">HOST</span>=${<span class="org-variable-name">1</span>:?Host not specified}; <span class="org-builtin">shift</span>
<span class="org-variable-name">DIR</span>=${<span class="org-variable-name">1</span>:?Remote directory not specified}; <span class="org-builtin">shift</span>
<span class="org-variable-name">PREFIX</span>=$(<span class="org-sh-quoted-exec">git rev-parse --show-prefix</span>)
<span class="org-variable-name">worktree</span>=$(<span class="org-sh-quoted-exec">git stash create</span>)
<span class="org-variable-name">worktree</span>=${<span class="org-variable-name">worktree</span>:-HEAD}
git push -f $<span class="org-variable-name">HOST</span>:$<span class="org-variable-name">DIR</span> $<span class="org-variable-name">worktree</span>:refs/heads/git-sarah
<span class="org-comment-delimiter"># </span><span class="org-comment">Use ssh to translate possibly relative $DIR to the absolute one.</span>
<span class="org-variable-name">ABS_DIR</span>=$(<span class="org-sh-quoted-exec">ssh "$HOST" "cd '$DIR' && pwd"</span>)
<span class="org-comment-delimiter"># </span><span class="org-comment">Run the command at $HOST and translate its output so that</span>
<span class="org-comment-delimiter"># </span><span class="org-comment">file/directory names appear as on localhost. Use perl's \Q to limit</span>
<span class="org-comment-delimiter"># </span><span class="org-comment">the possibility of interpretating ABS_DIR as a search pattern.</span>
ssh <span class="org-string">"$HOST"</span> <span class="org-string">"cd '$DIR/$PREFIX' && git -c advice.detachedHead='' checkout --detach git-sarah && ${*:?Command not specified}"</span> <span class="org-sh-escaped-newline">\</span>
|& perl -pe <span class="org-string">"s|\Q${ABS_DIR}\E|$(</span><span class="org-sh-quoted-exec">git rev-parse --show-toplevel</span><span class="org-string">)|g"</span>
</pre>
</div>
InstallFest 2014: Automatizace bootování s novabootemhttps://rtime.felk.cvut.cz/~sojka/blog/if14/2019-04-28T13:59:59Z2014-03-04T08:47:30Z
<p>
Stejně jako <a href="https://rtime.felk.cvut.cz/~sojka/blog/if13">vloni</a> jsem měl i letos přednášku na <a href="http://installfest.cz/if14/">InstallFestu</a>.
Tentokrát to bylo o nástroji <a href="https://github.com/wentasah/novaboot">novaboot</a>, který už přes dva roky vyvíjím a
používám jej k mnoha zajímavým věcem. Některé z nich jsem během
přednášky zmínil a najdete je ve <a href="https://rtime.felk.cvut.cz/~sojka/blog/if14/novaboot-if14.pdf">slidech</a>.
</p>
<p>
Při přednášce se dostavil klasický “demo efekt”, tedy že to, co ještě
ráno fungovalo, fungovat přestalo. Nemohl za to novaboot, ale pochybná
implementace technologie <a href="https://en.wikipedia.org/wiki/Intel_Active_Management_Technology">Intel AMT</a> na mém počítači. Naštěstí jsem měl
ještě jeden počítač, se kterým novaboot fungoval, ale bohužel jsem na
něj neměl otočenou web kameru, takže se návštěvníci museli spokojit
jen s výstupem na mém terminálu.
</p>
<div>
Slidy k přednášce:<br />
<a href="https://rtime.felk.cvut.cz/~sojka/blog/if14/novaboot-if14.pdf"><img src="https://rtime.felk.cvut.cz/~sojka/blog/if14/novaboot-if14.pdf-r75.png" width="378" height="284" alt="Slidy k přednášce" class="pdfimg" /></a>
</div>
On generating Linux applications from Simulinkhttps://rtime.felk.cvut.cz/~sojka/blog/on-generating-linux-applications-from-simulink/2019-04-28T13:51:35Z2013-07-25T15:36:07Z
<p>
I use Matlab/Simulink R2012b under Linux to model dynamic systems. In
order to perform hardware-in-the-loop testing, I want to run my models
in real-time on Linux with rt_preempt patch. Running a Simulink model
in real-time can be achieved by converting the model to the C code,
compiling it and running the resulting application. Simulink supports
this by means of <i>Simulink Coder</i>. Producing a Linux application with
Simulink Coder is not as straightforward as one might think, though.
</p>
<div id="outline-container-orgf4e22ef" class="outline-2">
<h2 id="orgf4e22ef">About Simulink Coder</h2>
<div class="outline-text-2" id="text-orgf4e22ef">
<p>
<a href="http://www.mathworks.com/products/simulink-coder/index.html">Simulink Coder</a> (formerly called Real-Time Workshop) can generate the
code in many styles and for several platforms and operating systems.
Surprisingly, Linux is also among <a href="http://www.mathworks.com/products/simulink-coder/description2.html">supported environments</a>.
</p>
<p>
Simulink Coder can, by default, generate code for so called <i>Generic
Real-Time Target</i> (GRT). Such code should be suitable for running on
desktop platforms. If you want to run the code on a more constrained
embedded system or you have additional requirements on the code (e.g.
compliance with safety standards), there is <a href="http://www.mathworks.com/products/embedded-coder/">Embedded Coder</a> for you.
Embedded Coder extends Simulink Coder with additional capabilities and
targets. The default Embedded Coder target is called <i>Embedded
Real-Time Target</i> (ERT).
</p>
<p>
As it turns out, neither GRT nor ERT target alone can be used to
create Linux executable even if documentation says the opposite.
</p>
</div>
</div>
<div id="outline-container-org4c4d0c0" class="outline-2">
<h2 id="org4c4d0c0">Available options for generating Linux applications</h2>
<div class="outline-text-2" id="text-org4c4d0c0">
<p>
From the documentation it seems that there are several options
available out of the box that allow to generate Linux applications
from Simulink. They are:
</p>
<ol class="org-ol">
<li><p>
The use of Embedded Real-Time target (<code>ert.tlc</code>) and selecting
<i>NativeThreadExample</i> as the target operating system in <i>Model
Configuration → Code Generation → Templates</i>.
</p>
<p>
When I tried to generate the code, I got this error:
</p>
<blockquote>
<p>
This model specifies the ‘Target operating system’ as
‘NativeThreadsExample’ from the ‘Template’ section of the ‘Code
Generation’ pane of the Configuration parameters dialog. This example
target is currently supported only when the model is configured for
concurrent execution.
</p>
</blockquote>
<p>
After some searching, I figured out how to enable <i>concurrent
execution</i>. It can be done at <i>View → Model Explorer → right click
on Configuration → Show Concurrent Execution options</i> and then
check <i>Allow tasks to execute concurrently on target</i>. Even though
I enabled this and configure the tasks, I still got the same error.
</p></li>
<li>The second option might be using the <i>IDE Link Target</i> that allows
to compile the generated code by Eclipse IDE. Even though I don’t
like Eclipse, I tried to set it up according to the documentation.
Again no success. I ended up with some strange errors suggesting
that there is a problem with Eclipse (more precisely its JVM) being
a 32 bit application and Matlab a 64 bit application. My guess is
that this feature is tested only on Windows. Since Matlab R2012b is
not distributed for 32 bit Linux, it is not possible to test this
option with 32 bit Matlab. Furthermore, in release notes of R2013a
it is announced that Eclipse support will be dropped in future
releases.</li>
</ol>
</div>
</div>
<div id="outline-container-org2ff3cb8" class="outline-2">
<h2 id="org2ff3cb8">Custom target for Linux applications</h2>
<div class="outline-text-2" id="text-org2ff3cb8">
</div>
<div id="outline-container-orga718854" class="outline-3">
<h3 id="orga718854">Simple target</h3>
<div class="outline-text-3" id="text-orga718854">
<p>
Since I failed with both above options I decided to write a custom
code generation target for Linux. One option would be to use <a href="http://sourceforge.net/projects/lintarget/">a target
previously developed at our department</a>, but it has several drawbacks:
</p>
<ol class="org-ol">
<li>The generated code cannot be compiled. It seems that the interface
to the generated code changed between Simulink version from 2007
(when the target was developed) and R2012b used by me.</li>
<li>I don’t like that the target is meant to be used on Windows
(hardcoded backslashes etc.) and additionally,</li>
<li>it contains features that I do not need (CANOpen block set).</li>
</ol>
<p>
Therefore I decided to write my own target that should be as simple as
possible.
</p>
<p>
After creating the target by copying the <code>ert.tlc</code> file and modifying
it slightly (basically replacing <code>ert</code> with <code>ert_linux</code>) and selecting
<i>NativeThreadsExample</i> as in the step 1 above, the code was
successfully generated. You can see this target <a href="https://rtime.felk.cvut.cz/gitweb/ert_linux.git/tree/8f90f30d7be0af2c82e51608ea08e07529e49496">here</a>. Such a target
has two problems:
</p>
<ol class="org-ol">
<li>It does not support <i>external mode</i>, i.e. a mode in which Simulink
can connect to the running application to provide on-line parameter
tuning and real-time data plotting.</li>
<li>The generated code uses POSIX timers that use signals to wake up
the threads. Signals have quite big overhead and there are rumors
that their delivery can suffer from priority inversion.</li>
</ol>
<p>
The only way to get Linux application with support for external mode
seemed to develop a target with custom <i>main.c</i> file.
</p>
</div>
</div>
<div id="outline-container-org339a444" class="outline-3">
<h3 id="org339a444">Advanced target</h3>
<div class="outline-text-3" id="text-org339a444">
<p>
I developed the target that provides the <i>main.c</i> file (actually
called <code>ert_main.c</code>) and supports external mode. Although the
documentation is quite extensive (several thousands pages in total),
if was really hard to figure out how to do it properly. Several times
it was necessary to study awful TLC<sup><a id="fnr.1" class="footref" href="https://rtime.felk.cvut.cz/~sojka/tags/linux/#fn.1" role="doc-backlink">1</a></sup> source code which is, IMHO,
very badly structured. Fortunately, I could use <a href="http://sourceforge.net/projects/lintarget/">Lukáš Hamáček’s</a> target
as an example, which helped me a lot.
</p>
<p>
The result of my work is can be <a href="https://rtime.felk.cvut.cz/gitweb/ert_linux.git">downloaded from our repository</a>. The
target has the following features:
</p>
<ul class="org-ul">
<li>Support for external mode over TCP.</li>
<li>Support for both single-rate and multi-rate systems.</li>
<li>Simulation threads are scheduled as <code>SCHED_FIFO</code> (real-time
priority).</li>
<li>External mode communication runs in non-real-time thread.</li>
</ul>
<p>
I tested the target on several systems and everything seemed to work
as expected. My only concern now is about thread safety. Given that
Simulink produces the error messages about concurrent execution of
threads (see above), I’m not sure whether the generated code can be
safely run in multiple threads. Also the code implementing external
mode server contains several semaphores but only when it is compiled
for the VxWorks OS. I’ll probably check this later if we encounter any
problems that might be caused by the code not being thread safe.
</p>
</div>
</div>
</div>
<div id="outline-container-org34d4e68" class="outline-2">
<h2 id="org34d4e68">Conclusion</h2>
<div class="outline-text-2" id="text-org34d4e68">
<p>
It is surprising how few relevant results Google gives when one asks
for ”<a href="https://www.google.cz/search?q%3Dsimulink%2Breal-time%2Bworkshop%2Blinux">simulink real-time workshop linux</a>”. It seems that the <a href="https://rtime.felk.cvut.cz/gitweb/ert_linux.git">Embedded
Coder target for Linux</a> described in this post might be useful for
people waning to run their Simulink models on Linux. Although that
the target may not work perfectly and in all configurations, I think
it is now good enough for interested people to try. If you experience
any problems with the target, let me know.
</p>
</div>
</div>
<div id="footnotes">
<h2 class="footnotes">Footnotes: </h2>
<div id="text-footnotes">
<div class="footdef"><sup><a id="fn.1" class="footnum" href="https://rtime.felk.cvut.cz/~sojka/tags/linux/#fnr.1" role="doc-backlink">1</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
Target Language Compiler – the tool used to convert Simulink
blocks to C code.
</p></div></div>
</div>
</div>
InstallFest 2013: Jak na Linux v robotechhttps://rtime.felk.cvut.cz/~sojka/blog/if13/2019-04-28T13:59:59Z2013-03-04T11:25:29Z
<p>
Na letošním <a href="http://installfest.cz/if13/">InstallFestu</a> jsem měl přednášku o použití Linuxu
v robotech. Snažil jsem se ukázat, jak lze z Linuxu běžícím na nějaké
“embedded” desce ovládat hardware k ní připojený. Všechny praktické
ukázky, které jsem měl připravené, se povedly, až na tu poslední –
robot v akci. Ale tu jsem nahradil <a href="https://www.youtube.com/watch?v=I-c59wIbAvo">videem z finále Eurobotu 2010</a>.
</p>
<div>
Slidy k přednášce:<br />
<a href="https://rtime.felk.cvut.cz/~sojka/blog/if13/if13-robot-linux.pdf"><img src="https://rtime.felk.cvut.cz/~sojka/blog/if13/if13-robot-linux.pdf-r75.png" width="378" height="284" alt="Slidy k přednášce" class="pdfimg" /></a>
</div>
<p>
<b>Update</b>: Video záznam z přednášky je k dispozici na <a href="http://installfest.cz/if13/archiv">stránkách
InstallFestu</a>.
</p>
Linuxhttps://rtime.felk.cvut.cz/~sojka/linux/2014-04-15T19:35:10Z2005-10-13T11:49:12Z
<h1>Tips & Tricks</h1>
<h2>Rename files in the current directory to short unique names</h2>
<pre><code>ls|rename '$_ = $Main::number++'
</code></pre>
<h2>GNOME</h2>
<p>If you create a custom keyboard shortcut
(<em>System->Preferences->Keyboard Shortcuts</em>) for executing a command,
the working directory of the command is set to /. If you do not like
it, as I do with starting my Emacs, you can use the following command
to change the directory:</p>
<pre><code>/bin/sh -c "cd $HOME; /usr/bin/emacsclient -c -a ''"
</code></pre>
<h2>Debian</h2>
<ul>
<li><p>Get the list of manually installed packages</p>
<pre><code>aptitude search '?installed?not(?automatic)' | awk '{print $2}'
</code></pre></li>
</ul>
<h2>Valkyrie</h2>
<ul>
<li><p>How to invoke emacs from valkyrie? Set Src Editor in Options to:</p>
<pre><code> /usr/bin/emacsclient -n -a emacs +%n
</code></pre></li>
</ul>
<h2>Pavel Píša’s BDM driver</h2>
<p><a href="http://cmp.felk.cvut.cz/~pisa/m683xx/bdm_driver.html">BDM driver</a></p>
<h3>How to set access permissions</h3>
<p>Create file <code>/etc/udev/rules.d/m683xx-bdm</code> with the following contents:</p>
<pre><code>ACTION=="add", SUBSYSTEM=="m683xx-bdm", OWNER="sojka"
DEVPATH=="/class/m683xx-bdm/icd0", SYMLINK="bdm"
</code></pre>
<p>Replace “sojka” at the first line with your login name or use
something like GROUP=”users” or MODE=”0666”. The second line
accomplishes the creation of symlink <code>/dev/bdm</code> for your default
device.</p>
<h2>X11</h2>
<h3>Login to remote server with local X terminal</h3>
<ul>
<li><p>If there is a remote display manager supporting XDMP and you are not
behind NAT, start X server from command line</p>
<pre><code> X -query <host>
</code></pre></li>
<li><p>If you are behind NAT, you can tunnel X connection through ssh. From
text console shell start:</p>
<pre><code> xinit /usr/bin/ssh <host> startkde -- :1
</code></pre></li>
</ul>
<p> This will start KDE remotely on second (:1) local display.</p>
<ul>
<li><p>To automate the previous command using KDM you can put the following
to your ~/.xsession</p>
<pre><code> ssh <host> startkde
</code></pre></li>
</ul>
<p>By selecting <strong>custom</strong> in the list of available X sessionsin in KDM,
this file will be automatically executed on login. If you have you ssh
keys protected by a password and ssh-askpass installed, you will be
automatically asked for the password to the key.</p>
<h1>Various Scripts</h1>
<ul>
<li><a href="https://rtime.felk.cvut.cz/~sojka/linux/novell">Mount Novell disks on DCE servers</a></li>
<li><a href="http://rtime.felk.cvut.cz/darcs/darcsweb.cgi?r=sojka/scripts;a=tree">Darcs repository of useful scripts</a></li>
</ul>