Below are notes for my Nix workshop at Installfest. https://pretalx.installfest.cz/installfest-2022/talk/QDTPQL/

1 Why Nix?

https://nixos.org/

  • Reproducible
    • Build exactly the same software on different systems.
  • Declarative
    • Store your development environment in a simple file/git.
  • “Reliable”
    • Nix ensures that installing or upgrading one package cannot break other packages.

2 Basics

2.1 Commands

  • Classic: nix-build, nix-shell, nix-env
    • older, incoherent, …
  • Experimental: nix
    • nix build, nix develop, nix profile
    • needs to be explicitly enabled:
      • nix --experimental-features nix-command flake
      • config file: ~/.config/nix/nix.conf

2.1.1 Package management

apt Nix classic Nix (flakes see later)
apt install mc nix-env -iA nixpkgs.mc nix profile install nixpkgs#mc
dpkg -l nix-env -q nix profile list
apt remove mc nix-env -e mc nix profile remove …
apt upgrade mc nix-env -uA nixpkgs.mc nix profile upgrade ..
  • Features:
    • No root required
    • Installed into ~./nix-profile/bin
ls -l ~/.nix-profile
ls -lH ~/.nix-profile
  • Updating

    nix-channel --list   # channel = package repository
    nix-channel --update # similar to apt update
    nix-env --upgrade    # upgrade installed packages to channel version
    

2.1.2 Searching

apt Nix classic Nix (flakes see later)
apt-cache search … nix-index + nix-locate nix search nixpkgs …
  (3rd party tool)  

https://search.nixos.org/packages

3 Under the hood

3.1 Nix store

ls -l /nix/store | head
ls -lh $(which bash)
tree $(dirname $(readlink $(which bash)))/.. | head

3.2 Profiles

https://nixos.org/manual/nix/stable/package-management/profiles.html

ls -l /nix/var/nix/profiles/per-user/$USER
  • Easy to switch to older versions
nix-env --rollback
nix-env --list-generations
nix-env --switch-generation 43
  • Comparing generations (experimental nix)

    nix profile diff-closures
    

4 Nixpkgs

4.1 Structure

  • pkgs – packages (programs/libraries/…)
  • nixos – Linux distribution based on nixpkgs
  • lib – helper functions
  • pkgs/top-level/all-packages.nix – list of top-level “attributes”
  • Searching: search for “attrname<space>=”

    cd nixpkgs
    grep -r 'gnuplot =' .
    

4.2 Building/Modifying packages

nix-build '<nixpkgs>' -A gnuplot
git clone https://github.com/NixOS/nixpkgs/
nix-build ./nixpkgs -A gnuplot
edit ./nixpkgs/pkgs/tools/graphics/gnuplot/default.nix
nix-build ./nixpkgs -A gnuplot
ls -l result
nix log result/  # show build log
nix log /nix/store/mr9l6qjimaarcak7s5rk6j1h4b16rpw9-gnuplot-5.4.3
# See the log from “distro” build
nix log $(dirname $(readlink $(which bash)))/..
cd nixpkgs
# Rebuild all packages depending on
nixpkgs-review wip

4.2.1 Nix output monitor

Let’s have better understanding what happens during building:

NIXPKGS_ALLOW_UNFREE=1 \
nix-build $HOME/.cache/nixpkgs-review/rev-877f0e97eb2eff62e675b75dfe19940400b2ed09-dirty/build.nix \
          --arg pkgs 'import <nixpkgs> {}' |& nom

Unfortunately does not work with experimental nix command, but will probably implement this in the future.

4.2.2 Examining dependencies

nix-tree ./result/

5 Development environments

Task Nix classic Nix
Temporary package installation nix-shell -p nix shell
Enter build environment of a package nix-shell nix develop

5.1 Install packages temporarily

mc --version
echo $PATH
echo $PATH | tr : \\n
nix-shell -p mc
mc --version
echo $PATH | tr : \\n
exit
nix-shell --pure -p mc  # only explicitly mentioned dependencies are available
echo $PATH | tr : \\n
exit
nix-shell -p mc -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/d1c3fea7ecbed758168787fe4e4a3157e52bc808.tar.gz

5.1.1 Experimental nix, flakes

  • Discoverability, CLI completion
nix shell nixpkgs#gnuplot
nix run nixpkgs#gnuplot

5.2 Existing package development/hacking

  • Traditionally:

    ./configure
    sudo apt install ...
    ./configure
    sudo apt install ...
    ./configure
    sudo apt install ...
    
  • With nix:

    git clone https://github.com/MidnightCommander/mc.git
    cd mc
    ./autogen.sh
    nix-shell '<nixpkgs>' -A mc
    ./autogen.sh
    ./configure
    make -j$(nproc)
    

5.3 Per-project development environments

  • One does not need to relay only on packages in nixpkgs
  • Source can contain Nix files (e.g. default.nix, shell.nix, flake.nix)
git clone https://github.com/wentasah/boardproxy
cd boardproxy
make  # fails due to missing dependencies
nix-shell  # make deps from shell.nix or default.nix available
make  # success
  • Just building the package:

    nix-build
    
    • Drawback: every single change in the source code rebuilds everything

5.3.1 Entering the environment automatically

6 How are Nix derivations built?

  • Nix expression is evaluated → low-level derivation

    nix-build '<nixpkgs>' -A gnuplot
    ls -l result
    nix-store --query --deriver result/
    nix show-derivation result/  # pretty format
    cat "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"
    
  • Important attributes: builder, args
  • Most attributes (and their values) are propagated to environment variables
nix-shell '<nixpkgs>' -A gnuplot
printenv
type genericBuild
type buildPhase
type configurePhase
unpackPhase
cd gnuplot-5.4.3
configurePhase
buildPhase
  • invalidace hash (aby se projevil update)

7 Generating Docker images from Nix packages

7.1 Running

nix-build docker-image.nix
docker load < result
docker run -it gnuplot:nmg0lnkf1dl7a0bsv152a6xphkxm9sq9

7.2 What’s inside?

mkdir tmp && cd tmp
tar xf ../result
ls -lR
tar tf */layer.tar

7.3 Image from custom package

{ pkgs ? import <nixpkgs> {}
, boardproxy ? import ./boardproxy/default.nix { inherit pkgs; }
}:

pkgs.dockerTools.buildImage {
  name = "boardproxy";
  config = {
    Entrypoint = "${boardproxy}/bin/boardproxy";
  };
}
nix-build docker-boardproxy.nix
docker load < result
docker run -it boardproxy:60j25jpwggspx1mg694143ddchsynq1q --help

7.4 Smaller image needed?

https://nixos.org/#asciinema-demo-cover

{ pkgs ? import <nixpkgs> {}
}:
pkgs.redis.overrideAttrs (old: {
  # no need for systemd support in our docker image
  makeFlags = old.makeFlags ++ ["USE_SYSTEMD=no"];
  # build static binary with musl
  preBuild = ''
     makeFlagsArray=(PREFIX="$out"
                     CC="${pkgs.musl.dev}/bin/musl-gcc -static"
                      CFLAGS="-I${pkgs.musl.dev}/include"
                      LDFLAGS="-L${pkgs.musl.dev}/lib")
    '';
  # Let's remove some binaries which we don't need
  postInstall = "rm -f $out/bin/redis-{benchmark,check-*,cli}";
})

8 Flakes intro

8.1 Problems of clasic Nix

  • What is inside a repo? Difficult to figure out without reading all *.nix files.
  • No enforcment of naming .nix files (default.nix, shell.nix, …)
  • What does <nixpkgs> mean? Depends on the value of $NIXPATH env. var.
  • nix-env operation depends on the content of ~/.nix-defexpr/.
  • Evaluation of Nix expressions can read arbitrary files on your disk. Harms reproducibility.

8.2 How do flakes solve it?

  • Hermetic isolation of Nix evaluation (no NIXPATH, no arbitrary files).
    • Only files from the same git repo can be read.
    • => Makes it possible to cache results of evaluation.
  • Defines a structure of repository “entry point” (flake.nix).
    • Predefined attribute names for packages, overlays, tests, …
  • Explicit references with versions to other sources, e.g. nixpkgs.
    • lock files

8.3 Drawbacks of flakes

  • One more thing to learn.
  • Lower motivation to include software into nixpkgs (can be separate flake)

9 Other notes

10 Pros and cons

  • Nix vs. Docker
    • Docker solves problems of traditional package managers by building on top of them
    • Nix changes (fixes) the core of package management
  • Nix is (largely) reproducible
  • Declarative specifications of build/development/production environments
  • Nix is difficult to learn
  • Using unpackaged applications (especially proprietary) is more difficult
    • buildFHSUserEnv
    • nix-ld
    • autopatchelf-hook
Posted Sun Jun 12 23:03:50 2022

Below are the materials from my today’s Installfest talk. https://pretalx.installfest.cz/installfest-2022/talk/Z8TP3D/

1 Characteristics

https://julialang.org/

  • New language – announced 2012
  • Goal: Solve the “two language” problem
    • Prototype in Python, rewrite in C++ for speed
  • Compiled language (LLVM)
  • Good for interactive work (JIT)
  • Dynamic types + type inference
  • Garbage collected
  • Multiple dispatch paradigm
    • one function can have different implementations based on type of all arguments
  • Lisp-like macros
  • Target domain:
    • Originally: Scientific computations
    • Nowadays: General purpose, Data Science, Visualisation, …

1.1 Speed

1.2 My success story

  • Load & process 2GB CSV file
  • Julia: 40s, 5 GB of memory
  • Python Pandas: 5 min, OUT OF MEMORY
  • Python Pandas + big machine: 15 min, 36 GB of memory

2 Installation

  • distribution package managers
  • download archive, unzip/untar, run
  • console application julia-terminal.png

2.1 IDE: Julia plugin for VS Code

3 Calculator

1 + 1
3 * 5

4 REPL modes

Special characters at the start of the line switch modes:

  • Help

    ?atan
    
  • Shell

    ;ls
    
  • Package manager

    ]add IJulia
    

Return with backspace

5 Variables

x = 1
x

# Julia likes Unicode :-)
π  # Enter as \pi<TAB>
α = atan(1/2)

3α + 100 # You can skip multiply operator

6 Vectors, Arrays

a=[10,20,30]
a[1]  # one-based indexing
a[2:3]
a[2:end]

b=[1 2; 3 4]

c=[5 6
   7 8]

6.1 Element-wise operations – broadcasting

a=rand(10_000_000)
b=rand(10_000_000)

a*b                 # 
a.*b
sin(a)              # 
sin.(a)             # Broadcasting
[sin(i) for i in a] # Comprehension

# Broadcasting is faster – no need to
# allocate intermediate memory
@time sin.(a) .+ cos.(b);
@time [sin(i) for i in a] .+ [cos(i) for i in b];

# In-place assignment (no memory allocation)
c = zeros(size(a))
@time c .= sin.(a).^2 .+ cos.(b).^2;
# Tired of writing dots?
@. c = sin(a)^2 + cos(b)^2

6.2 GPU arrays

https://juliagpu.org/

  • Available APIs
    • CUDA.jl
    • AMDGPU.jl
    • oneAPI.jl (Intel)
  • High-level (array) API:

    using oneAPI
    oneAPI.versioninfo()
    a=rand(100_000_000)
    b=rand(100_000_000)
    c=zeros(size(a))
    @time c .= a .* b;
    ga = oneArray(a)
    gb = oneArray(b)
    gc = oneArray(c)
    @time gc .= ga .* gb;
    
  • Possibility to write GPU kernels in Julia

7 Types

Julia uses dynamic types + type inference.

typeof(1)
typeof(1.0)
typeof(Int8(1))
typeof("Hello world")

7.1 Parametric types

1//3
typeof(ans)    # ans = result of previous line
typeof(Int8(1)//Int8(3))
typeof([1,2,3])
typeof([1 2 3])
typeof(Int8[1,2,3])
typeof([1 2;3 4.1])

7.2 Abstract types and type hierarchy

“is subtype” operator: <:

Integer <: Number
Int8 <: Integer
Int8 <: Number
Float32 <: Number
Float32 <: Integer

7.3 User-defined types

mutable struct MyType
    id::Int64
    text::String
end

x = MyType(10, "Hello")

# Define addition of our type
Base.:+(a::MyType, b::MyType) = MyType(a.id + b.id, "Sum")
x + MyType(1, "xxx")

8 Functions

# Mathematics-like definition
square(x) = x*x
square(4)

# Programming-like definition
function hello(name)
    return "Hello " * name
end
hello("Michal")

square("Hello")

# Operators are functions too
+(1,2)
→(a,b) = (a+b)/(a-b)
3→1

8.1 Anonymous functions

Functional programming features.

x->x^2
ans(5)  # call

a=rand(10)
# Return elements > 0.5
filter(x->x>0.5, a)
[x for x in a if x > 0.5]
a[a .> 0.5]

8.2 Methods and multiple dispatch

Functions can have multiple methods (implementations). The method to call is selected based on types of all arguments.

fun(x::Number) = "Number "*string(x)
methods(fun)
fun(1)
fun(1.1)
fun("1.1")
fun(x::String) = "String "*x
methods(fun)
fun("1.1")

# Functions can have many methods
methods(+)
methods(show)

8.3 Multimedia I/O (show examples)

Values can be presented in different formats:

@doc atan
typeof(@doc atan)
show(stdout, MIME("text/plain"), @doc atan)
show(stdout, MIME("text/html"), @doc atan)

9 Macros

  • Inspired by Lisp
  • Work at abstract syntax tree level.
  • Can rewrite/generate programs.
    • Often used for domain specific languages
  • No separate macro language, macros are written in Julia itself.

9.1 Examples

9.1.1 Unit testing

using Test
@test 1 + 1 == 2

9.1.2 Code simplification

sqrt(abs(sin(1)))

# Pipe syntax (for unary functions only)
1 |> sin |> abs |> sqrt

rnd = rand(10)
sort(rnd, rev=true) .+ 1

# Pipes with higher-arity functions ⇒ lambdas
rnd |>  x -> sort(x, rev=true) |> x -> x .+ 1
using Pipe

# Piped value represented by underscore
@pipe rnd |> sort(_, rev=true) |> _ .+ 1
  • Similar: Chains.jl, DataFramesMeta.jl, …
@chain df begin
  dropmissing
  filter(:id => >(6), _)
  groupby(:group)
  combine(:age => sum)
end

9.2 Benchmarking, code inspection, optimization

@time rand(Int(1e6));

using BenchmarkTools
@benchmark rand(Int(1e6))

@code_native sum(1:5)

a = [1, 2, 3]
# Don’t perform bounds checking
@inbounds a[2]

10 Showcase

Example of what is possible with the language. This is not builtin functionality. Everything is programmed in Julia.

10.1 Measurements

Computation with confidence intervals

using Measurements
a = 5.2 ± 1.0
typeof(a)
b = 3.7 ± 1.0
a + b
a * b
a / b

10.2 Unitful

using Unitful
using Unitful.DefaultSymbols
using Unitful: hr

1m + 3cm |> cm |> float
sin(90)
sin(90°)
sin(π/2)

15m/3s

10km/hr |> m/s
10km/hr |> m/s |> float
0°C |> K |> float

11 Plotting

  • Plots supports multiple plotting backends (e.g. Python matplotlib).
  • Infamous “Time to first plot” – much better today
using Plots
plot(sin.(0:0.1:2π))
  • One of several available interfaces to Gnuplot.
  • Faster than Plots
using Gnuplot
@gp sin.(0:0.1:2π) "with lines lw 5" "set grid"

12 Package management

  • Fast development, breaking changes (packages with version < 1.0)
  • Reproducible environments (projects)
    • Which packages and versions
    • Project.toml, Manifest.toml
  • Needs manual setup

12.1 Package manager

pwd()
# Switch to package manager
]
?
# create a new project or activate existing
activate .
add Pipe
;cat Project.toml
cat Manifest.toml

12.2 Using packages, modules

  • Module = namespace

    module MyMod
    export x
    x=1
    y=2
    end
    x
    MyMod.x
    
    • Package is a git repo with certain structure
    # introduce exported symbols to current namespace
    using Package
    # introduce just the symbol Package  to current namespace
    import Package
    

13 Tasks & Channels

  • Easy to use parallelism
  • Similar to goroutines in Go

14 Interfacing other languages

Direct call to a function from shared library:

# libc call
ccall(:clock, Int32, ())
# using other libraries
ccall((:zlibVersion, "libz"), Cstring, ()) |> unsafe_string

Python code can be called transparently from Julia:

using PyCall
math = pyimport("math")
math.sin(math.pi / 4) # returns ≈ 1/√2 = 0.70710678...

15 Interactive notebooks

  • Jupyter vs. Pluto.jl
    • Pluto is something between Jupyter and Excel

15.1 Jupyter

using IJulia
notebook()

15.2 Pluto.jl

import Pluto
Pluto.run()

See ./installfest-pluto.html

15.3 Feature comparison

Feature Jupyter Pluto.jl
Languages many Julia
File format JSON Julia script with comments
Results Stored in JSON Available only at runtime
Execution order Top-down/manual Dependency-based
Cell updates Manual Automatic
Package management No Yes, reproducible

16 Dataframes.jl

  • Work with tabular data, named columns
  • Easy import from CSV (CSV.jl)

17 Conclusion

  • ➕ “Simple”, fast, versatile language
  • ➕ A lot of packages available
  • ➕ Active community
  • ➖ Some packages are not mature, breaking changes
  • ➖ Compilation can be slow (new session)
Posted Sun Jun 12 22:55:57 2022

Recently, I worked on a https://github.com/CTU-IIG/kcf. The project has several compile configurations and a time-consuming test suite, some parts of which need a GPU not present in my laptop. Because many parts of the code depend heavily on compile configuration, changing them requires recompilation and ideally also retesting of all configurations to make sure that everything works correctly. Since my development laptop not only lacks the GPU needed for testing but also has only four cores, which makes the compilation unnecessarily slow, I looked for an easy way to compile and test the code being edited on a powerful remote server.

Initially, I tried to use git commit and git push to capture the state of the working directory and transfer it to the remote server, but this was not ideal for many reasons. Then I discovered git stash create, which captures the state of the working directory and creates a “volatile” commit object not attached to any branch. This command became the basis of my git-sarah script, which is an acronym standing for git stash and run at host, and you can see find its code below.

With git-sarah script, instead of instructing my editor or IDE to run:

make

I instruct it to run the following:

git sarah example.org dir -- make

This command copies the current state of the work tree to $HOME/dir on the example.org server and runs make in the directory corresponding to the current directory on the local host. For example, if the current local directory is ~/projects/kcf/src, with kcf being the root directory of the git repository, the make command will be run in $HOME/dir/src on the server. In the output (both stdout and stderr) of the command, the path of the server directory, e.g., /home/mylogin/dir is replaced with its local counterpart: /home/me/projects/kcf.

If a single command (make) is not enough, you can instruct git-sarah to run commands from a script (e.g., commands.sh), which need not be stored in your git repository, by running:

git sarah example.org dir -- sh < ./commands.sh

The code of git-sarah is quite simple:

#!/bin/bash

set -e -o pipefail

OPTS_SPEC="\
git sarah <host> <remote_dir> [ -- ] [ command ... ]

git stash and run at host
--
h,help    show the help
"
eval "$(echo "$OPTS_SPEC" | git rev-parse --parseopt -- "$@" || echo exit $?)"
echo "$*"
while \[[ $# -gt 0 ]]; do
    case "$1" in
        -h) exit 0;;
        --) shift; break;;
    esac
    shift
done

HOST=${1:?Host not specified}; shift
DIR=${1:?Remote directory not specified}; shift
PREFIX=$(git rev-parse --show-prefix)

worktree=$(git stash create)
worktree=${worktree:-HEAD}

git push -f $HOST:$DIR $worktree:refs/heads/git-sarah

# Use ssh to translate possibly relative $DIR to the absolute one.
ABS_DIR=$(ssh "$HOST" "cd '$DIR' && pwd")

# Run the command at $HOST and translate its output so that
# file/directory names appear as on localhost. Use perl's \Q to limit
# the possibility of interpretating ABS_DIR as a search pattern.
ssh "$HOST" "cd '$DIR/$PREFIX' && git -c advice.detachedHead='' checkout --detach git-sarah && ${*:?Command not specified}" \
    |& perl -pe "s|\Q${ABS_DIR}\E|$(git rev-parse --show-toplevel)|g"
Posted Sat May 11 00:04:24 2019 Tags:

Transcript of my seminar about https://julialang.org/.

Calculator

1 + 1
3 * 5

Variables

x = 1
x
π  # Enter as \pi<TAB>
α = atan(1/2)

3α + 100

Vectors, Arrays

a=[10,20,30]
a[1]
a[2:3]
a[2:end]

b=[1 2; 3 4]

Element-wise operations – broadcasting

a=rand(100_000_000)
b=rand(100_000_000)

a*b
a.*b
sin(a)
sin.(a)
[sin(i) for i in a]

@time sin.(a) .+ cos.(b);
@time [sin(i) for i in a] .+ [cos(i) for i in b];

c = zeros(size(a))
@time c .= sin.(a).^2 .+ cos.(b).^2;
@. c = sin(a)^2 + cos(b)^2

REPL modes

  • Help

    ?atan
    @doc atan
    
  • Shell

    ;ls
    
  • Package manager

    ]add IJulia
    

Return with backspace

Types

Julia uses dynamic types + type inference.

typeof(1)
typeof(1.0)
typeof(Int8(1))
typeof("Hello world")

Parametric types

1//3
typeof(ans)
typeof(Int8(1)//Int8(3))
[1,2,3]
[1 2 3]
Int8[1,2,3]
[1 2;3 4.1]

Abstract types and type hierarchy

Integer <: Number
Int8 <: Integer
Int8 <: Number
Float32 <: Number
Float32 <: Integer

Functions

f(x) = x*x
f(4)
function hello(name)
    return "Hello " * name
end
hello("Michal")
f("Hello")

+(1,2)
→(a,b) = (a+b)/(a-b)
3→1

Anonymous functions

x->x^2
ans(5)
a=rand(10)
filter(x->x>0.5, a)

[x for x in a if x > 0.5]

a[a .> 0.5]

Methods and multiple dispatch

Function can have multiple implementations. The implementation to call is selected based on argument types

f(x::Number) = "Number "*string(x)
methods(f)
f(1)
f(1.1)
f("1.1")
f(x::String) = "String "*x
f("1.1")
methods(+)
methods(show)

Show examples

@doc atan
typeof(@doc atan)
show(stdout, MIME("text/plain"), @doc atan)
show(stdout, MIME("text/html"), @doc atan)

typeof(`ls`)
run(`ls`)

Units

Example of what is possible with the language. This is not a builtin functionality. Everything is programmed in Julia.

using Unitful
using Unitful.DefaultSymbols
using Unitful: hr

1m + 3cm |> cm |> float
float(cm(1m + 3cm))
1m + 3cm |> cm |> float
sin(90)
sin(90°)
sin(π/2)

15m/3s

10km/hr |> m/s
10km/hr |> m/s |> float
0°C |> K |> float

Plotting

Plots supports multiple plotting backends.

using Plots
pyplot()
plot(sin.(0:0.1:2π)

One of several available interfaces to Gnuplot.

using Gaston
plot(sin.(0:0.1:2π)

Macros

  • Inspired by Lisp
  • Work at parse tree level.
  • Can rewrite/generate programs.
  • No separate macro language, macros are written in Julia itself.

Examples (see test suites of many Julia packages):

@testset
@test

Tasks & Channels

  • Easy to use parallelism

Jupyter notebooks

using IJulia
notebook()

Integration with C/C++

Direct call to a function from shared library:

ccall(:clock, Int32, ())
ccall((:clock, "libc"), Int32, ())

Interactive C++

using Cxx

cxx"#include <stdio.h>"
icxx"""printf("Hello world\n");"""
# Passing Julia values to C/C++ functions 
x = 42
@cxx printf(pointer("X is %d\n"), x)

Complex example

Testing a C++ class and plotting results of its work.

using Cxx
import Cxx.CxxCore
using Gaston

cxx""" #include "stepper.cpp" """

# Create C++ object and store it in Julia variable
s = icxx"Stepper(nullptr, 2020, 100, 20);"
# Call a method of C++ object
@cxx s->moveTo(2000)
pos = Int32[0]
freq = Int32[0]
while @cxx s->isMoving()
    @cxx s->periodic()
    # Store values from the C++ object in Julia array for later
    # plotting
    push!(freq, @cxx s->freq)
    push!(pos, @cxx s->getPosition())
end

# Plot gathered data
set(grid="on")
x=0:length(freq)-1
plot(x, freq, xlabel="iteration", legend="freq")
plot!(x, [0; pos[2:end] .- pos[1:end-1]]./20e-3, plotstyle="points", color="red", legend="Δpos/T_s")
plot!(x, pos, plotstyle="linespoints", color="green", legend="pos")

# Cxx cannot (yet) change functions once that are defined. Use these
# to commands to “replace” the compiler object, which allows
# redefining the functions.
Cxx.CxxCore.reset_init!()
Cxx.new_clang_instance()
Posted Fri Apr 26 23:45:50 2019 Tags:

When studying Linux kernel sources, it’s often helpful to use ftrace’s function_graph tracer, which traces function entry and exit points and clearly shows how functions call each other.

Sometimes, it is useful to get such a trace from other code than the Linux kernel. In case of C++ this https://stackoverflow.com/a/21863962/902448, but for plain C, it is trickier. I came up with the following to print nice traces. It uses https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#Common-Variable-Attributes to register a function exit callback.

#include <stdio.h>

int trace_indent = 2;
char trace_filler[] = {[0 ... 99] = ' '};

static void trace_end(const char **func)
{
    trace_indent--;
    fprintf(stderr, "%.*s}\n", 4*trace_indent, trace_filler);
}

#define TRACE(format, ...) \
    const char *__tracer __attribute__ ((__cleanup__(trace_end))) = __func__; \
    fprintf(stderr, "%.*s%s(" format ") {\n", 4*trace_indent, trace_filler, __func__, ##__VA_ARGS__); \
    trace_indent++;

It can be used this way:

void bar()
{
        TRACE();
}

void foo(char *str)
{
        TRACE("str=%s", str);
        bar();
}

int main(int argc, char *argv[])
{
        TRACE("argc=%d", argc);
        foo("xxx");
        return 0;
}

When this program is run, it prints:

main(argc=1) {
    foo(str=xxx) {
        bar() {
        }
    }
}
Posted Sat Aug 25 09:51:02 2018 Tags:

„Slidy“ z mého workshopu na Installfestu. K dispozici i ve formě ./org-mode.txt, jen je potřeba změnit příponu z .txt na .org.

About

  • ASCII by mělo být spíš UTF-8, ale to nezní tak „cool“ :-)
  • O mě
    • Pracuji na ČVUT, FEL
    • Embedded systémy (≈ low-level stuff)
    • Emacs používám od r. 1998
    • Patřím k lidem, kteří v Emacsu žijí
    • Vim používám také a nic proti němu nemám :-)

Org-mode

  • Pokud je Emacs označován jako operační systém, org-mode bychom mohli nazvat kancelářský balík
  • Outliner
    • Základní funkcionalita; snaha je, aby další funkce nekomplikovaly použití org-mode jako outlineru.
  • Značkovací jazyk (markup)
  • Prostředí pro přípravu textových dokumentů
  • Tabulkový kalkulátor
  • Kalendář, plánovač, organizér
  • a určitě mnoho dalšího…
  • Dokumentace: «C-h i d m org RET»
  • Nekonečné možnosti konfigurace
  • «M-x customize-group RET org RET»

Outliner

  • Nadpisy ^*+
    • M-RET – nový nadpis
    • M-šipky, M-S-šipky – změny úrovně či pořadí
    • C-c C-u – o úroveň výš, C-c C-j – skok, org#Motion
    • Schovávání: TAB, S-TAB
  • Seznamy
    • odrážkové, číslované, …
    • S-←,→
    • Mnohé klávesové zkratky stejné jako u nadpisů
  • Konverze «C-c *», «C-c -»

Značkovací jazyk (markup)

  • Tučné, kurzíva, podtrženo verbatim, přeskrtnuto, kód α,
  • Tabulky – nejsnadnější zadávaní jednoduchých tabulek do počítače | xxx | yyy «TAB»
  • Obrázky: [[./file.jpg]​]
  • Příklady, zdrojáky se zvýrazňováním syntaxe
    • Zkratky: <e «TAB», <s «TAB»
Příklad
print("String " + number)

Odkazy

  • [[cíl]​[popis]] nebo [​[cíl]]
  • https://installfest.cz/if17/ «C-c C-l», «C-c C-o»
  • Cíle odkazů
    • URL
    • ./soubor
    • interní odkaz [​[cíl]]

      • * Nadpis obsahující cíl
      • #+NAME: cíl

      3.3.1

    • Externí odkazy («C-c l»)
      • [[file:source.c::main(int]] argc
      • (tetris)
      • odkazy na emaily…
  • „Rádiové cíle“: <<<rádio>>> rádio

    Příklad: Rádio bylo vynalezeno roku 1893.

Organizace

  • Todo – každý nadpis může být položkou „TODO“ «S-←,→»
  • Seznam stavů je konfigurovatelný
  • «C-c a t» – all TODOs = agenda
  • Seznam souborů pro tvorbu agendy (např. jeden .org soubor pro projekt)
  • Scheduled/deadline: C-c C-s, C-c C-d
    • «.» = dnes, «S-šipky»
    • Rozumí i textu: např. „May 1“ nebo „Wed“
  • Záznam odpracovaného času na úkolech «C-c C-x C-i», «C-c C-x C-o»
  • Souhrn splnění podúkolů [/], [%]
  • [ ] Checkboxy «C-c C-c»
  • Tagy u nadpisů :tag:, «C-c C-c»
    • hledání podle tagů: «C-c a m»

Rychlý záznam TODO (capture)

  • Vyžaduje konfiguraci
    • Globální klávesa «C-c c»
    • Šablony - předvyplněný obsah, kam se má „TODO“ uložit
(setq org-capture-templates
 '(("t" "Todo" entry (file+headline "~/org/gtd.org" "Tasks")
        "* TODO %?\n  %i\n  %a")
   ("j" "Journal" entry (file+datetree "~/org/journal.org")
        "* %?\nEntered on %U\n  %i\n  %a")))

Export

  • «C-c C-e»
  • Plain text, HTML, OpenDocument, …
  • Relativně snadná tvorba vlastních exportů (např. Doxygen)
  • Nastavení exportu v hlavičce – «C-c C-e #»

Babel

  • Práce se zdrojovými kódy
    • Podpora více jazyků v jednom dokumentu a interakce mezi nimi
  • „Literate programming“
  • „Reproducible research“ (executable documentation)
for i in $(seq 3); do date; sleep 1; done
ls -l

Výsledky vidíme níže:

Wed  8 Mar 10:44:56 CET 2017
Wed  8 Mar 10:44:57 CET 2017
Wed  8 Mar 10:44:58 CET 2017
total 24
-rw-r--r-- 1 wsh wsh 6696 Mar  5 21:52 graph.png
-rw-r--r-- 1 wsh wsh 6232 Mar  8 10:42 index.org
-rw-r--r-- 1 wsh wsh 6232 Mar  8 10:44 org-mode.txt
  • Editace: «C-c ’»
  • Spuštění: «C-c C-c»
  • Argumenty v hlavičce
    • :results verbatim drawer

Babel example

Run the benchmark (shell script) on a remote server:

mbw -a -q 10 | awk '{ print $1, $3, $9 }'
0 MEMCPY 2193.945
1 MEMCPY 2334.267
2 MEMCPY 2368.546
3 MEMCPY 2396.358
4 MEMCPY 2411.382
5 MEMCPY 2417.211
6 MEMCPY 2434.867
7 MEMCPY 2436.054
8 MEMCPY 2433.682
9 MEMCPY 2460.025
0 DUMB 2494.388
1 DUMB 2513.826
2 DUMB 2511.301
3 DUMB 2508.781
4 DUMB 2506.894
5 DUMB 2510.67
6 DUMB 2508.151
7 DUMB 2518.892
8 DUMB 2475.86
9 DUMB 2465.483
0 MCBLOCK 3248.863
1 MCBLOCK 3172.589
2 MCBLOCK 3161.555
3 MCBLOCK 3175.611
4 MCBLOCK 3184.713
5 MCBLOCK 3174.603
6 MCBLOCK 3164.557
7 MCBLOCK 3170.577
8 MCBLOCK 3166.561
9 MCBLOCK 3188.776

Gnuplot code to plot the results of the above command.

set key right bottom Left
set grid
set title "mbw results"
set xlabel "Attempt"
set ylabel "Bandwidth [MiB/s]"
set style data lp
set yrange [0:]
plot "< grep MEMCPY ".data  using 1:3 title "MEMCPY", \
     "< grep DUMB ".data    using 1:3 title "DUMB", \
     "< grep MCBLOCK ".data using 1:3 title "MCBLOCK"

graph.png

Properties

  • «C-c C-x p»
  • Key-value přiřazené k nadpisu
  • Může měnit různé nastavení pro danou sekci
  • Lze používat i jako jednoduchou databázi (hledání podle properties apod.)
  • Column view «C-c C-x C-c»
Posted Wed Mar 8 11:06:41 2017 Tags:

I started using http://kicad-pcb.org/ (version 4.0.1) for electronic design. It seems to be a great tool, but working with big connectors in schema editor is tedious. All connections have to be done manually and thus the process is error prone. Some way of automating the process would help. KiCad comes with http://confluence.kicad-pcb.org/display/KICAD/KiCad+Scripting+Reference+Manual, but unfortunately only in pcbnew. Eeschema seems not to have any support for automation yet. Hopefully, one can use http://www.semicomplete.com/projects/xdotool/ under Linux to automate some boring tasks.

I wanted to attach the same labels to FMC connector pins as on the http://zedboard.org/sites/default/files/documentations/MBCC_FMC_UG_1.1.pdf – there is about 70 pins used. My initial state was this:

before.png

I copied the pin names and net labels from the http://zedboard.org/sites/default/files/documentations/MBCC-FMC_RevB_Schematic_140211.pdf and put together the script bellow, which takes care of attaching the labels to the right pins. So after a few seconds I got the following:

after.png

The Python script that does the magic is here:

#!/usr/bin/env python3

import subprocess

conn = [
    ("LA06_P", "C10"),
    ("LA06_N", "C11"),
    ("LA10_P", "C14"),
    ("LA10_N", "C15"),
    ("LA14_P", "C18"),
    ("LA14_N", "C19"),
    ("LA18_CC_P", "C22"),
    ("LA18_CC_N", "C23"),
    ("LA27_P", "C26"),
    ("LA27_N", "C27"),

    ("LA01_CC_P", "D08"),
    ("LA01_CC_N", "D09"),
    ("LA05_P", "D11"),
    ("LA05_N", "D12"),
    ("LA09_P", "D14"),
    ("LA09_N", "D15"),
    ("LA13_P", "D17"),
    ("LA13_N", "D18"),
    ("LA17_CC_P", "D20"),
    ("LA17_CC_N", "D21"),
    ("LA23_P", "D23"),
    ("LA23_N", "D24"),
    ("LA26_P", "D26"),
    ("LA26_N", "D27"),

    ("CLK1_M2C_P", "G02"),
    ("CLK1_M2C_N", "G03"),
    ("LA00_CC_P", "G06"),
    ("LA00_CC_N", "G07"),
    ("LA03_P", "G09"),
    ("LA03_N", "G10"),
    ("LA08_P", "G12"),
    ("LA08_N", "G13"),
    ("LA12_P", "G15"),
    ("LA12_N", "G16"),
    ("LA16_P", "G18"),
    ("LA16_N", "G19"),
    ("LA20_P", "G21"),
    ("LA20_N", "G22"),
    ("LA22_P", "G24"),
    ("LA22_N", "G25"),
    ("LA25_P", "G27"),
    ("LA25_N", "G28"),
    ("LA29_P", "G30"),
    ("LA29_N", "G31"),
    ("LA31_P", "G33"),
    ("LA31_N", "G34"),
    ("LA33_P", "G36"),
    ("LA33_N", "G37"),

    ("CLK0_M2C_P", "H04"),
    ("CLK0_M2C_N", "H05"),
    ("LA02_P", "H07"),
    ("LA02_N", "H08"),
    ("LA04_P", "H10"),
    ("LA04_N", "H11"),
    ("LA07_P", "H13"),
    ("LA07_N", "H14"),
    ("LA11_P", "H16"),
    ("LA11_N", "H17"),
    ("LA15_P", "H19"),
    ("LA15_N", "H20"),
    ("LA19_P", "H22"),
    ("LA19_N", "H23"),
    ("LA21_P", "H25"),
    ("LA21_N", "H26"),
    ("LA24_P", "H28"),
    ("LA24_N", "H29"),
    ("LA28_P", "H31"),
    ("LA28_N", "H32"),
    ("LA30_P", "H34"),
    ("LA30_N", "H35"),
    ("LA32_P", "H37"),
    ("LA32_N", "H38"),
]

subprocess.call(["xdotool", "search", "--name", "/FMC/", "windowfocus"])
xtab = str.maketrans({"_": "underscore"})
for net in conn:
    label = [char.translate(xtab) for char in net[0]]
    pin = [char.translate(xtab) for char in net[1]]
    subprocess.call(["xdotool", "key", "--delay", "50", "ctrl+f"] + pin + ["Return", "Escape", "Down", "Left", "Left", "Left", "w",
                     "Left", "Left", "Left", "Left", "Left", "Left", "Left", "Left",
                     "Left", "Left", "k", "l"] + label + [ "Return", "Return"])
Posted Sun Jan 10 15:03:28 2016 Tags:

Osnova, podle které jsem jel na dnešním workshopu na Installfestu.

1 Co je Emacs?

  • Všestranný textový editor
  • Interpret Lispu (v jazyce C) + spousta Lispového kódu
    • C: 280 kLoC (20%), Lisp: 1 MLoC (80%)
    • Srovnání – Firefox: 6 MLoC C/C++, 3.6 MLoC JavaScript+HTML
  • Emacs = Editor MACroS nebo Esc-Meta-Alt-Ctrl-Shift :)
  • Plnohodnotný operační systém :) (M-x tetris, M-x doctor)
  • Začal vznikat v 70. letech

emacs_learning_curves.png

2 Terminologie

  • Poplatná době vzniku
  • Buffer = otevřený soubor
  • Okno (window) = část obrazovky editoru zobrazující buffer
    • Jedno okno zobrazuje právě jeden buffer
    • Jeden buffer může být zobrazován nula až N okny
  • Frame = to čemu dneska říkáme okno
    • Může být grafický nebo textový (terminál)

3 Základní klávesové zkratky

  • Mnoho jich je stejných jako v bashi
  • Dobře se používají i na exotických klávesnicích (kámošův notebook, BT klávesnice k mobilu/tabletu, …)
  • Notace: C = Control, M = Meta = Alt = Esc, S = Shift

3.1 Kurzor

  • C-b = backward char, C-f = forward char, C-p = prev line, C-n = next line
  • M-b = backward word, M-f = forward word, M-p = prev ???, M-n = next ???
  • C-a = home, C-e = end
  • M-a = backward sentence, M-e = forward sentence

3.2 Mazání

  • C-d = znak, M-d = slovo
  • C-k = do konce řádky

3.3 Příkazy

  • C-x C-f = find (open) file
  • C-x C-s = save file
  • C-x C-c = “exit”
  • C-x u, C-_, C-/ = undo

Pozn.: C-x je tzv. prefixová klávesa, podobně jako Esc

  • C-l = redisplay, center, …
  • C-g = přeruší právě vykonávanou operaci (mačkat, když Emacs nereaguj)

3.4 Hledání

  • C-s = interactive search, C-M-s = interactive regexp search
  • C-r = reverse search

3.5 Prefix/numeric argument

  • Neplést s prefixovou klávesou
  • Mnohým příkazům (klávesovým zkratkám) lze předat číselný parametr
  • Většinou to znamená opakování
  • C-u 8 C-f – posun o 8 znaků doprava
  • M-8 C-f – to samé
  • C-u samotné modifikuje některé příkazy i bez čísla (C-u C-x C-c = vše ulož a “exit”)

4 Příkazová řádka

  • M-x <příkaz>
  • Mini-buffer – většina klávesových zkratek stejná jako při editaci normálních bufferů (souborů)
  • tab-completion
  • completion “modes”
  • Příkaz je lispovská funkce označená jako (interactive).
  • Klávesové zkratky spouští příkazy – viz (global-set-key ...).

5 Velké (major) a malé (minor) módy

  • V Emacsu se vše motá kolem “módů”
  • Velký mód:
    • definuje zvýrazňování syntaxe,
    • definuje klávesové zkratky,
    • implementuje (automatické) zarovnávání kódu,
    • vybírá se automaticky podle přípony nebo obsahu souboru,
    • lze ho vybrat i manuálně (spuštěním příkazu *-mode; např. c-mode).
  • Malý mód:
    • Implementuje dodatečnou funkcionalitu, kterou má smysl používat s vícero velkými módy.
    • Příklady:
      • flyspell-mode
      • auto-fill-mode
      • hide-show-mode
  • Jaké módy jsou momentálně aktivní naznačuje Mode line.

6 Přístup k nápovědě

  • Emacs má poměrně dobrý systém nápovědy.
  • C-h prefix key
    • C-h f – funkce
    • C-h v – proměnné
    • C-h k – klávesová zkratka
    • C-h m – mód
    • C-h i – prohlížeč info stránek (C-h i m Emacs <RET>)
      • Navigace: ‘n’, ‘p’, ’[’, ’]’
      • ’i’ – hledání v rejstříku (pak ’,’)
  • M-x man

7 Bloky (region, rectangle)

  • Nové verze Emacsu (konečně) podporují výběr pomocí S-<šipky>.
  • Tradičně: blok je oblast od “značky” ke kurzoru (point).
  • C-SPC = polož značku
  • Ctrl-C, Ctrl-V, Ctrl-X – standardně nefunguje, lze zapnout M-x cua-mode.
    • Ctrl-C koliduje s mnoha standardními zkratkami; cua-mode to částečně řeší.
  • Tradičně:
    • Smazaný text se ukládá do “kill ringu” (např. C-k, M-d)
    • C-y (yank) vloží hodnotu z konce “kill ringu”
    • M-y (následující po C-y) vkládá starší hodnoty z “kill ringu”.
    • C-w = kill region
    • M-w = “copy” region
  • Obdélníkové bloky
    • C-x r * – neintuitivní (aspoň pro mne)
    • C-x SPC = rectangle mark mode
      • M-x string-insert-rectangle – vloží stejný text na každou řádku bloku
    • CUA mode: C-RET = cua-set-rectangle-mark
      • string-insert-rectangle není potřeba; stačí začít psát
      • ve výjimečných případech se chová podivně

8 Správa oken a bufferů

  • Správa oken
    • není ideální
    • C-x 2, C-x 3 = rozděl okno vertikálně/horizontálně
    • C-x o = přesun na následující okno
    • Windmove package – pohyb mezi okny podle směru (např. S-<šipky>)
    • C-x 0 = smaž aktuální okno (ale ne buffer)
    • C-x 1 = smaž ostatní okna
  • Přepínání bufferů
    • C-x <left>/<right>
    • C-x b = zadat název bufferu (použitelné s completion módy)
    • C-x C-b (polo) interaktivní výběr bufferu
    • M-x ibuffer = interaktivní výběr bufferu
    • M-x helm-buffer-list = dtto

9 Užitečné příkazy/klávesové zkratky

  • M-q = přeformátování odstavce (v mnoha módech inteligentní)
  • C-s = interactive search, C-M-s = interactive regexp search
    • V příkazové řádce hledání:
      • C-s – minulé hledání
      • C-w – doplní slovo pod kurzorem
      • M-c – přepne rozlišování malých a velkých písmen
      • M-e – editace (např. minulého hledání po C-s C-s)
  • M-% = nahrazení řetězce
  • C-M-S-% = nahrazení regulárního výrazu
  • M-/ = doplň rozepsané slovo podle předchozích slov
  • Macros: F3 = začátek nahrávání, F4 = konec nahrávání/spuštění
  • Nahrazení textu ve víc souborech najednou: M-x dired-mode, mark (m), Q
  • M-x occur
  • M-x rgrep
  • M-; comment-dwim (do what I mean)
  • C-M-a = beginning-of-defun, C-M-e = end-of-defun
  • C-M-p, C-M-n, C-M-u = pohyb po závorkami ohraničených blocích (prev, next, up)
  • C-u C-SPC – návrat na místa předchozí editace

10 Konfigurace, “Customization”

  • Možnosti konfigurace jsou nekonečné
  • Předkonfigurovaný Emacs: https://github.com/bbatsov/prelude
  • Standardní konfigurační soubor ~/.emacs nebo ~/.emacs.d/init.el.
    • Nastavování proměnných
    • Konfigurace klávesových zkratek (global-set-key (kbd "C-c w") 'toggle-truncate-lines)
    • Definice vlastních příkazů
    • Zapínání minor módu
  • Customization
    • User Hacker-friendly konfigurace
    • +GUI+TUI pro konfiguraci skoro všeho
    • M-x customize
    • M-x customize-{variable,face,group}
    • Není potřeba ručně editovat ~/.emacs

11 File/directory-local variables

  • Konfiguraci Emacsu lze ukládat i do editovaných souborů.
  • Podobně je možné konfiguraci uložit do souboru .dir-locals.el a pak bude platit pro všechny soubory v daném adresáři a jeho podadresářích.
  • M-x add-file-local-variable
  • M-x add-dir-local-variable
/* Local Variables: */
/* compile-command: "make CFLAGS='-g -O0 -Wall" */
/* End: */

12 Programování

  • Odsazování
    • většinou automatické, nebo po <TAB>
    • inteligentní zarovnávání všeho
      • ’M-q’ v komentáři zarovná komentář
  • Kompilace
    • M-x compile, M-x recompile
    • M-x next-error (skáče na chyby, detekce mnoha kompilátorů/formátů chybových hlášení)
  • Doplňování kódu
  • Často možnost interakce s interpretrem skriptovacích jazyků (např. v M-x python-shell-send-defun)
  • Verzovací systémy
    • vc – stejné klávesové zkratky pro všechny VCS (např. C-x v = – diff aktuálního souboru)
    • magit – speciální mód pro práci s gitem

13 Užitečné módy

  • https://www.gnu.org/software/auctex/ – pokročilý editor na *TeXové dokumenty
    • navigace po sekcích, promote/demote sekcí
    • integrace s BibTeXem pro snadné hledání/vkládání citací
    • náhledy rovnic a obrázků přímo v editovaném textu
  • http://orgmode.org/ – tvorba a organizování poznámek, TODO seznamů, plánování projektu, tvorba dokumentů atd.
    • tato prezentace je v org módu
  • dired-mode – správa souborů
    • C-x d, C-x C-f (adresář)
    • mazání souborů: označit ‘d’, smazat ‘x’
  • TRAMP
    • vzdálená editace (kompilace, …) souborů
    • SSH, sudo, …
  • http://notmuchmail.org/ – lokální “gmail” (rychlé vyhledávání), GUI v Emacsu
  • shell, eshell – práce s příkazovou řádkou bez opuštění Emacsu

Instalace balíků třetích stran:

14 emacsclient

  • Emacs většinou dlouho startuje
  • Doba startu záleží na uživatelské konfiguraci
    • Inicializace balíků při startu vs. autoload
  • Emacs lze pustit jako “server” (emacs --daemon) a připojovat se k němu pomocí programu emacsclient.
    • Nic se nemusí konfigurovat, emacsclient si spustí emacs sám:

      #!/bin/sh
      exec emacsclient -a '' "$@"
      
  • K jednomu serveru se lze připojit současně z různých terminálů (lokálně X-windows, vzdáleně přes SSH).

15 Úvod do Emacs Lispu

  • (Emacs) Lisp
    • C-h i m Elisp <RET>
    • Je jednodušší než např. Common Lisp
    • Program i data vypadají stejně (S-výrazy/S-expressions/sexp)
    • Vyhodnocení výrazu:
      • číslo je číslo, řetězec je řetězec
      • seznam: první prvek udává jméno funkce, ostatní prvky jsou parametry.

        (message "Ahoj")
        (message (concat "Ahoj" "Installfest"))
        
    • Proměnné:
      • globální:

        (defvar xyz 0)
        
      • lokální:

        (let (a b)
          <kód>)
        
    • Funkce:

      (defun fs-czech ()
        "Starts flyspell with czech dictionary"
        (interactive) ; Způsobí, že fs-czech se dá spustit pomocí M-x
        (flyspell-mode)
        (ispell-change-dictionary "czech")
        (flyspell-buffer))
      

16 Příklad použití elispu

  • Navigace po framech v LaTeX beameru

    (defconst latex-beamer-frame-re         ; Regulární výraz, který budeme hledat
      "\\\\\\(frametitle\\|section\\){")    ; '\' se musí 2x escapovat
    
    (defun is-outside-of-comment ()
      (null (nth 8 (syntax-ppss))))
    
    (defun latex-beamer-previous-frame ()
      (interactive)
      (while (progn
               (re-search-backward latex-beamer-frame-re)
               (not (is-outside-of-comment))))
      (recenter 0))
    
    (defun latex-beamer-next-frame ()
      (interactive)
      (when (looking-at-p latex-beamer-frame-re)
        (forward-char))
      (while (progn
               (re-search-forward latex-beamer-frame-re)
               (not (is-outside-of-comment))))
      (goto-char (match-beginning 0))
      (recenter 0))
    
    (add-hook 'LaTeX-mode-hook (lambda ()
      (local-set-key (kbd "M-p") 'latex-beamer-previous-frame)
      (local-set-key (kbd "M-n") 'latex-beamer-next-frame)))
    

17 In-line elisp

  • C-x C-e = vyhodnoť výraz před kurzorem (point)
  • Kusy elispu lze vkládat do komentářů programů v jiných jazycích a spouštět přímo tam.
\chapter*{List of Acronyms}

% Sort acronyms: (let ((sort-fold-case t)) (sort-regexp-fields nil "^[[:space:]]*\\\\acro\{\\([^}]*\\)\}.*$" "\\1" (search-forward "\\begin{acronym}") (search-forward "\\end{acronym}")))
\begin{acronym}[XXXXX]
  \acro{AC}{Access Category}
  \acro{ACK}{Acknowledge}
  \acro{BFQ}{Budget Fair Queuing}
\end{acronym}
Posted Sun Mar 8 17:10:35 2015 Tags:

Stejně jako file:if13 jsem měl i letos přednášku na http://installfest.cz/if14/. Tentokrát to bylo o nástroji https://github.com/wentasah/novaboot, který už přes dva roky vyvíjím a používám jej k mnoha zajímavým věcem. Některé z nich jsem během přednášky zmínil a najdete je ve ./novaboot-if14.pdf.

Při přednášce se dostavil klasický “demo efekt”, tedy že to, co ještě ráno fungovalo, fungovat přestalo. Nemohl za to novaboot, ale pochybná implementace technologie https://en.wikipedia.org/wiki/Intel_Active_Management_Technology na mém počítači. Naštěstí jsem měl ještě jeden počítač, se kterým novaboot fungoval, ale bohužel jsem na něj neměl otočenou web kameru, takže se návštěvníci museli spokojit jen s výstupem na mém terminálu.

Slidy k přednášce:
Slidy k přednášce
Posted Tue Mar 4 09:47:30 2014 Tags:

I use Matlab/Simulink R2012b under Linux to model dynamic systems. In order to perform hardware-in-the-loop testing, I want to run my models in real-time on Linux with rt_preempt patch. Running a Simulink model in real-time can be achieved by converting the model to the C code, compiling it and running the resulting application. Simulink supports this by means of Simulink Coder. Producing a Linux application with Simulink Coder is not as straightforward as one might think, though.

About Simulink Coder

http://www.mathworks.com/products/simulink-coder/index.html (formerly called Real-Time Workshop) can generate the code in many styles and for several platforms and operating systems. Surprisingly, Linux is also among http://www.mathworks.com/products/simulink-coder/description2.html.

Simulink Coder can, by default, generate code for so called Generic Real-Time Target (GRT). Such code should be suitable for running on desktop platforms. If you want to run the code on a more constrained embedded system or you have additional requirements on the code (e.g. compliance with safety standards), there is http://www.mathworks.com/products/embedded-coder/ for you. Embedded Coder extends Simulink Coder with additional capabilities and targets. The default Embedded Coder target is called Embedded Real-Time Target (ERT).

As it turns out, neither GRT nor ERT target alone can be used to create Linux executable even if documentation says the opposite.

Available options for generating Linux applications

From the documentation it seems that there are several options available out of the box that allow to generate Linux applications from Simulink. They are:

  1. The use of Embedded Real-Time target (ert.tlc) and selecting NativeThreadExample as the target operating system in Model Configuration → Code Generation → Templates.

    When I tried to generate the code, I got this error:

    This model specifies the ‘Target operating system’ as ‘NativeThreadsExample’ from the ‘Template’ section of the ‘Code Generation’ pane of the Configuration parameters dialog. This example target is currently supported only when the model is configured for concurrent execution.

    After some searching, I figured out how to enable concurrent execution. It can be done at View → Model Explorer → right click on Configuration → Show Concurrent Execution options and then check Allow tasks to execute concurrently on target. Even though I enabled this and configure the tasks, I still got the same error.

  2. The second option might be using the IDE Link Target that allows to compile the generated code by Eclipse IDE. Even though I don’t like Eclipse, I tried to set it up according to the documentation. Again no success. I ended up with some strange errors suggesting that there is a problem with Eclipse (more precisely its JVM) being a 32 bit application and Matlab a 64 bit application. My guess is that this feature is tested only on Windows. Since Matlab R2012b is not distributed for 32 bit Linux, it is not possible to test this option with 32 bit Matlab. Furthermore, in release notes of R2013a it is announced that Eclipse support will be dropped in future releases.

Custom target for Linux applications

Simple target

Since I failed with both above options I decided to write a custom code generation target for Linux. One option would be to use http://sourceforge.net/projects/lintarget/, but it has several drawbacks:

  1. The generated code cannot be compiled. It seems that the interface to the generated code changed between Simulink version from 2007 (when the target was developed) and R2012b used by me.
  2. I don’t like that the target is meant to be used on Windows (hardcoded backslashes etc.) and additionally,
  3. it contains features that I do not need (CANOpen block set).

Therefore I decided to write my own target that should be as simple as possible.

After creating the target by copying the ert.tlc file and modifying it slightly (basically replacing ert with ert_linux) and selecting NativeThreadsExample as in the step 1 above, the code was successfully generated. You can see this target https://rtime.felk.cvut.cz/gitweb/ert_linux.git/tree/8f90f30d7be0af2c82e51608ea08e07529e49496. Such a target has two problems:

  1. It does not support external mode, i.e. a mode in which Simulink can connect to the running application to provide on-line parameter tuning and real-time data plotting.
  2. The generated code uses POSIX timers that use signals to wake up the threads. Signals have quite big overhead and there are rumors that their delivery can suffer from priority inversion.

The only way to get Linux application with support for external mode seemed to develop a target with custom main.c file.

Advanced target

I developed the target that provides the main.c file (actually called ert_main.c) and supports external mode. Although the documentation is quite extensive (several thousands pages in total), if was really hard to figure out how to do it properly. Several times it was necessary to study awful TLC1 source code which is, IMHO, very badly structured. Fortunately, I could use http://sourceforge.net/projects/lintarget/ target as an example, which helped me a lot.

The result of my work is can be https://rtime.felk.cvut.cz/gitweb/ert_linux.git. The target has the following features:

  • Support for external mode over TCP.
  • Support for both single-rate and multi-rate systems.
  • Simulation threads are scheduled as SCHED_FIFO (real-time priority).
  • External mode communication runs in non-real-time thread.

I tested the target on several systems and everything seemed to work as expected. My only concern now is about thread safety. Given that Simulink produces the error messages about concurrent execution of threads (see above), I’m not sure whether the generated code can be safely run in multiple threads. Also the code implementing external mode server contains several semaphores but only when it is compiled for the VxWorks OS. I’ll probably check this later if we encounter any problems that might be caused by the code not being thread safe.

Conclusion

It is surprising how few relevant results Google gives when one asks for ”https://www.google.cz/search?q%3Dsimulink%2Breal-time%2Bworkshop%2Blinux”. It seems that the https://rtime.felk.cvut.cz/gitweb/ert_linux.git described in this post might be useful for people waning to run their Simulink models on Linux. Although that the target may not work perfectly and in all configurations, I think it is now good enough for interested people to try. If you experience any problems with the target, let me know.

Footnotes:

1

Target Language Compiler – the tool used to convert Simulink blocks to C code.

Posted Thu Jul 25 17:36:07 2013 Tags: