I just released a C++17 library to perform automatic differentiation: Mission : impossible
J’ai également, en français, une introduction à la différentiation automatique. Les exemples sont en Julia.
Easy Creation of GnuPlot Scripts from C++
Table of Contents
file:https://travis-ci.org/vincent-picaud/GnuPlotScripting.svg?branch=master
1 What is it?
A simple C++17 library that allows easy creation and execution of gnuplot scripts. These scripts will embed their data and can be replayed/modified later.
The library depends on {fmt} library.
I use it when I want to plot some data with a minimal effort when I develop stuff in C++.
For the moment the library is only tested under Linux (it should also works under Windows but I have not checked yet).
Feel free to use it: GitHub, GnuPlotScripting.
1.1 News
-
Added SVG, TGIF, PNGCairo and PDFCairo export formats. Tagged as v0.0.3. -
Some minor fixes + doc concerning Global_Config logger methods. Tagged as v0.0.2 -
Initial version. Tagged as v0.0.1
1.2 Contributors
- Guillaume Jacquenot: Travis CI
2 Installation
The library currently uses the meson build system.
If you are not familiar with meson, the compilation procedure is as follows:
git clone git@github.com:vincent-picaud/GnuPlotScripting.git cd GnuPlotScripting/ meson build cd build ninja test
Examples can then be found in the build/examples/ directory.
3 Examples
3.1 Plot and fit data
#include "GnuPlotScripting/GnuPlotScripting.hpp" #include <iostream> #include <vector> using namespace GnuPlotScripting; // From: https://www.cs.hmc.edu/~vrable/gnuplot/using-gnuplot.html int main() { std::vector<double> time, angle, stdvar; time = {0.0, 1.0, 2.1, 3.1, 4.2, 5.2, 6.2, 7.2, 8.2, 9.1, 10.0, 11.0, 12.0, 12.9, 13.8, 14.9, 15.9, 17.0, 17.9, 18.9, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0, 30.0, 31.0, 32.0, 32.9, 33.8, 34.7, 35.7, 36.6, 37.7}; angle = {-14.7, 8.6, 28.8, 46.7, 47.4, 36.5, 37.0, 5.1, -11.2, -22.4, -35.5, -33.6, -21.1, -15.0, -1.6, 19.5, 27.5, 32.6, 27.5, 20.2, 13.8, -1.3, -24.5, -25.0, -25.0, -20.2, -9.9, 5.8, 14.7, 21.8, 29.8, 21.4, 24.6, 25.8, 0.6, -16.6, -24.0, -24.6, -19.8}; stdvar = {3.6, 3.6, 3.0, 3.4, 3.5, 3.4, 10.3, 3.4, 3.4, 3.5, 3.6, 3.9, 3.9, 4.2, 2.7, 3.2, 2.8, 3.5, 2.7, 3.3, 3.4, 4.2, 6.7, 3.3, 3.1, 3.6, 3.2, 3.2, 3.0, 3.5, 2.7, 4.1, 2.7, 12.0, 2.9, 3.2, 3.7, 3.8, 3.5}; Script_File script("plot.gp"); Data_Vector data( time, angle, stdvar); // <- you can stack as many vector/valarray etc.. as you want // only size() and operator[] are required. script.free_form("set title 'Cavendish Data'"); script.free_form("set xlabel 'Time (s)'"); script.free_form("set ylabel 'Angle (mrad)'"); script.free_form("set grid"); script.free_form("plot {} with yerrorbars notitle", data); script.free_form("replot {} u 1:2 with lines title '{}'", data, "raw data"); script.free_form("theta(t) = theta0 + a * exp(-t / tau) * sin(2 * pi * t / T + phi)"); script.free_form("fit theta(x) {} using 1:2:3 via a, tau, phi, T, theta0", data); script.free_form("replot theta(x) lw {} lc {} title 'best-fit curve'", 2, 4); script.export_as(PNG(), "plot"); }
It generates this figure:
Note: the generated plot.gp gnutplot script embeds the data and you can replay it whenever you want:
gnuplot plot.pg -
3.2 Matrix data
#include "GnuPlotScripting/GnuPlotScripting.hpp" #include <iostream> using namespace GnuPlotScripting; // Example from: https://stackoverflow.com/a/27049991/2001017 // Also see: https://stackoverflow.com/q/32458753/2001017 // int main() { Data_Ascii data( "0.00 0.65 0.65 0.25\n" "0.25 0.00 0.75 0.25\n" "0.50 0.60 0.00 0.25\n" "0.75 0.25 0.10 0.00\n"); Script_File script("matrix.gp"); script.free_form("set autoscale fix"); script.free_form("set cbrange [-1:1]"); script.free_form("unset colorbox"); script.free_form("unset key"); script.free_form( "plot {} matrix using 1:2:3 with image, '' matrix using " "1:2:(sprintf('%.2f', $3)) with labels font ',16'", data); script.export_as(PNG(), "matrix"); script.export_as(EPSLATEX().set_standalone(true), "matrix"); }
It generates this figure:
It also generates a standalone matrix.tex file you can process with pdflatex matrix.tex to get a monochrome matrix.pdf file. If you want colorized pdf simply use:
EPSLATEX().set_standalone(true).set_color(true)
3.3 Histogram
#include "GnuPlotScripting/GnuPlotScripting.hpp" #include <iostream> #include <random> using namespace GnuPlotScripting; // Example from: // https://stackoverflow.com/a/7454274/2001017 // template <typename T> void gnuplot_histogram(Script& script, const std::vector<T>& data, const size_t n_bin, const typename std::vector<T>::value_type min, const typename std::vector<T>::value_type max) { assert(max > min); assert(n_bin > 0); Data_Vector gnuplot_data(data); const double width = (max - min) / n_bin; script.free_form("width={}", width); script.free_form("set title 'Histogram min={}, max={}, Δbin={}, #bins={}, #sample={}'", min, max, width, n_bin, data.size()); script.free_form("hist(x,width)=width*floor(x/width)+width/2.0"); script.free_form("set boxwidth width*0.9"); script.free_form("set style fill solid 0.5"); script.free_form("plot {} u (hist($1,width)):(1.0) smooth freq w boxes notitle", gnuplot_data); } int main() { std::random_device rd; std::mt19937 gen(rd()); const double a = 2, b = 1; std::gamma_distribution<> distribution(a, b); std::vector<double> data(10000); for (auto& data_i : data) data_i = distribution(gen); Script_File script("histogram.gp"); gnuplot_histogram(script, data, 100, 0, 3); script.export_as(PNG(), "histogram"); }
The generated figure is:
3.4 Graph
#include "GnuPlotScripting/GnuPlotScripting.hpp" #include <iostream> using namespace GnuPlotScripting; // Example from the "Gnuplot in Action" book int main() { Data_Ascii data( "-1 -1 0 # A\n" "-1 1 0 # B\n" " 1 0 0 # C\n" " 0 0 1.75 # D\n" "\n\n" "-1 -1 0 -1 1 0 \n" "-1 -1 0 1 0 0 \n" "-1 -1 0 0 0 1.750 \n" "-1 1 0 1 0 0 \n" "-1 1 0 0 0 1.75 \n" " 1 0 0 0 0 1.75 \n"); Script_File script_a("graph_3D.gp"); script_a.free_form("unset border"); script_a.free_form("unset tics"); script_a.free_form("unset key"); script_a.free_form("set view 75,35"); script_a.free_form("splot {} index 0 with points pointtype 7 pointsize 3", data); script_a.free_form("replot {} index 1 u 1:2:3:($4-$1):($5-$2):($6-$3) with vectors nohead", data); script_a.free_form("pause -1"); Script_File script_b("graph_2D.gp"); script_b.free_form("unset border"); script_b.free_form("unset tics"); script_b.free_form("unset key"); script_b.free_form("plot {} index 0 with points pointtype 7 pointsize 3", data); script_b.free_form("replot {} index 1 u 1:2:($4-$1):($5-$2) with vectors nohead", data); script_b.export_as(PNG(), "graph"); }
It generates this figure:
but also an active gnuplot 3D figure you can rotate etc…
3.5 Pipe example
Instead of creating a file, we can create a pipe with popen() to directly send data to gnuplot.
#include "GnuPlotScripting/GnuPlotScripting.hpp" #include <chrono> #include <iostream> #include <thread> #include <utility> // std::pair #include <vector> using namespace GnuPlotScripting; int main() { // AFAIK one has to replot all data at each iteration // std::vector<std::pair<size_t, double>> data; Script_Pipe pipe(Script_Pipe_Mode_Enum::Not_Persistent); pipe.free_form("set xlabel 'iterations'"); for (size_t i = 0; i < 100; i++) { data.push_back({i, 1 / (i + 1.)}); pipe.free_form("plot '-' using 1:2 with lines t \"residue\" "); for (const auto& data_i : data) { pipe.free_form("{} {}", data_i.first, data_i.second); } pipe.free_form("e"); pipe.flush(); std::this_thread::sleep_for(std::chrono::milliseconds(50)); } }
3.6 Supported export formats
This example silently exports a basic plot in all supported formats:
#include "GnuPlotScripting/GnuPlotScripting.hpp" #include <iostream> using namespace GnuPlotScripting; int main() { Script_File script("available_export_formats.gp", Script_File_Mode_Enum::Silent); script.free_form("plot sin(x) t 'sin(x)'"); script.export_as(PNG(), "available_export_formats"); script.export_as(EPSLATEX().set_standalone(true), "available_export_formats"); script.export_as(SVG(), "available_export_formats"); script.export_as(TGIF(), "available_export_formats"); script.export_as(PNGCairo(), "available_export_formats_cairo"); script.export_as(PNGCairo().set_color(false), "available_export_formats_cairo_nocolor"); script.export_as(PDFCairo(), "available_export_formats_pdfcairo"); }
3.7 Global config demonstration
This last example shows how to use Global_Config.
#include "GnuPlotScripting/GnuPlotScripting.hpp" #include <iostream> using namespace GnuPlotScripting; int main() { global_config().set_logger( [](const char *const msg) { std::cerr << "====> My logger " << msg << std::endl; }); // If you want to remove logger: global_config().set_logger(); // If you want to restore the default one: global_config().set_default_logger(); // If you want to globally overwrite Script_File_Mode_Enum to Persistent, do: global_config().set_script_file_mode(Script_File_Mode_Enum::Persistent); for (size_t i = 1; i < 5; i++) { Script_File script(fmt::format("script_{}.gp", i), Script_File_Mode_Enum::Silent); script.free_form("plot sin({0}*x) t 'sin({0}*x)'", i); } // Now switch back to a regular configuration: stop overwriting // local script_file_mode. global_config().set_script_file_mode(); // Now this will silently run scripts for (size_t i = 1; i < 5; i++) { Script_File script(fmt::format("script_{}.gp", i), Script_File_Mode_Enum::Silent); script.free_form("plot sin({0}*x) t 'sin({0}*x)'", i); } }
4 Documentation
The library is quite simple and there is only 3 things you must know:
- Data_XXX are classes to store your data
- Script_XXX are script classes to write your scripts
- global_config() returns a Global_Config object used to define global options.
4.1 Data classes
4.1.1 Data_Vector
Creates columns of data from std::vector, std::valarray… In fact only the size() method and the operator[] operator are used and you can use any object defining these two methods. By example:
std::vector<double> v1(10); std::vector<int> v2(10); std::valarray<double> v3(10); // ... Data_Vector data(v1,v2,v3);
4.1.2 Data_Ascii
uses data already put in form into a std::string. By example:
Data_Ascii data( "0.00 0.65 0.65 0.25\n" "0.25 0.00 0.75 0.25\n" "0.50 0.60 0.00 0.25\n" "0.75 0.25 0.10 0.00\n");
It is really easy to define your own Data class. By example Data_Ascii code is as simple as:
class Data_Ascii final : public Data { public: Data_Ascii(const std::string& data) : Data(data) {} };
Also note that the Data base class provides a uuid that insures that data is embedded only once in the gnuplot script.
By example, when you write:
script.free_form("plot {} u 1:2",data); script.free_form("replot {} u 1:3",data); script.free_form("replot {} u 1:4",data);
data is copied into the script file only once.
4.2 Script classes
There are two script classes:
Script_Filecreates a file to store the script.Script_Pipecreates a pipe to push data directly to GnuPlot, in that case no file is created.
They inherits from the Script base class that provides the following methods:
template <typename... ARGS> Script& free_form(ARGS&&... args); Script& export_as(const Export_As& export_as, const std::filesystem::path& output); void flush();
free_formallows you to write free form using thefmtlibrary, by example:
script.free_form("plot '{}' u {}:{}","data_file.dat",1,2);
flush()forces buffer to be flushedexport_as()generates script code to export the figure in the given format, by example:
script.export_as(EPSLATEX.set_standalone(true),"filename");
Note:
- the right extension for
filenameis automatically added
(here this would be .tex).
- for the moment only two export formats are defined
PNG,EPSLATEX. I will add more in the future.
4.2.1 Script_File class
The only relevant part is the constructor:
Script_File(const std::filesystem::path& filename, Script_File_Mode_Enum script_file_mode = Script_File_Mode_Enum::Persistent);
Filenameis the gnuplot script file name (you are free to use the file extension you want, on my side I use the.gpextension).script_file_modeis important as it defines what happens at destruction timeScript_File_Mode_Enum::Nonedoes nothingScript_File_Mode_Enum::Silentsilently runs GnuPlot (this will generate your exported figures)Script_File_Mode_Enum::Persistentruns GnuPlot in persistent mode, it will generates your figures and left a window opened that allows you to see the result. This is only an opened window and not an active gnuplot session (you cannot interact with the plot).
Note: to get an active GnuPlot session, you can replay your script with:
gnuplot filename.gp -
(note the final ‘-‘, see GnuPlot documentation for further details).
Another possibility is to add a pause in your gnuplot script:
script.free_form("pause -1");
4.2.2 Script_Pipe class
Here instead of writing into a file, we open a pipe with popen. This allows you to directly command GnuPlot during your code execution. Note that this is only a unidirectional channel.
The constructor is:
Script_Pipe(Script_Pipe_Mode_Enum script_pipe_mode = Script_Pipe_Mode_Enum::Persistent);
as for Script_File class, script_pipe_mode defines what happens at destruction time:
Script_Pipe_Mode_Enum::Not_Persistentdoes not keep an opened windowScript_Pipe_Mode_Enum::Persistentkeeps an opened, but inactive, window
4.3 Global_Config class
This class allows you to define or overwrite globally some options
const char* gnuplot_exe() const; Global_Config& set_gnuplot_exe(const char* const gnuplot_executable); Global_Config& set_logger(); // removes logger Global_Config& set_default_logger(); // reuses default one Global_Config& set_logger(const std::function<void(const char* const msg)>& f); // defines your own bool has_logger() const; Global_Config& set_log_message(const char* const msg); Global_Config& set_script_file_mode( Script_File_Mode_Enum mode); // globally overwrite local 'script_file_mode' Global_Config& set_script_file_mode(); // stop overwriting local 'script_file_mode' std::optional<Script_File_Mode_Enum> script_file_mode() const;
set/gnuplot_exe()functions allow you to define GnuPlot executable filename, by default this isgnuplotorgnuplot.exefor windows.set/logger()functions allow you to stop or redirect logs, by example:
global_config().set_logger([](const char *const msg) { std::cerr << "====> My logger " << msg << std::endl; });
-
set_script_file_mode()functions are more interesting as they allow you to overwrite globally what happens atScript_Filedestruction time. A typical use case is as follows:Imagine that your code silently generates a lot of scripts:
for (size_t i = 1; i < 5; i++) { Script_File script(fmt::format("script_{}.gp", i), Script_File_Mode_Enum::Silent); script.free_form("plot sin({0}*x) t 'sin({0}*x)'", i); }
However at debug time, you want to force visualization to see what happens. In that case you simply have to add
global_config().set_script_file_mode(Script_File_Mode_Enum::Persistent);
before
for (size_t i = 1; i < 5; i++) { ... }
This will force all Script_File to use Script_File_Mode_Enum::Persistent
5 References
- GnuPlot official page
- GnuPlot in Action a very well written book
- www.gnuplotting.org a lot of great examples
- GnuPlot not so Frequently Asked Questions
- Wikipedia the free encyclopedia…
6 FAQ
-> your question here
Mimicking named optional arguments in C++17
I just wrote a C++17 one header file library that will allow you to define named optional arguments.
It can be used to improves code readability by replacing obscure function/method calls like
algorithm(x_init,
100,
std::vector<double>(n, 0),
std::vector<double>(n, 1),
1e-6,
1e-6)
by something like
algorithm(x_init,
max_iterations = 100,
absolute_precision = 1e-6);
algorithm(x_init,
absolute_precision = 1e-6,
lower_bounds<double> = std::vector<double>(n, 0));
algorithm(x_init);
supporting all the variations in position/definition for the optional arguments.
More details can be found in the OptionalArgument GitHub repository.
Curiously Recurring Template Pattern (CRTP) in depth
Table of Contents
1 Classical implementation
CRTP is a useful tool to define C++ static interface (Wikipedia) CRTP.
As it does not induce run-time penalties, it is widely used when performance is required.
Maybe C++20’s Concepts will supplant this technique, but before I wanted to share my notes on the classical CRTP.
Its most basic form is as follows
#include <iostream> template <typename IMPL> class Crtp { public: const IMPL& impl() const { return static_cast<const IMPL&>(*this); }; size_t size() const { return impl().size(); }; }; class A1 : public Crtp<A1> // the CRTP trick { public: size_t size() const { return 10; }; }; class A2 : public Crtp<A2> // the CRTP trick { public: size_t size() const { return 20; }; }; template <typename IMPL> void foo(const Crtp<IMPL>& v) // same code for A1 & A2, // no run-time penalty. { std::cout << "\n" << v.size(); } int main() { A1 a1; foo(a1); A2 a2; foo(a2); }
10 20
As you can see, CRTP allows you to define generic code (the foo() function) without run-time penalty (everything can be inlined by the compiler).
In the next sections I use c++17, but everything can be adapted to c++14 (or even to c++11 if you ignore the decltype(auto)) without pain.
2 Specializing return type
A first observation is that if you forgot to define the size() method
class A2 : public Crtp<A2> { public: // size_t size() const // oppss // { // return 20; // }; };
then invoking a2.size() will cause a run-time error as
size_t Crtp::size() const { return impl().size(); // infinite recursion };
produces an infinite recursion (thus a stack overflow).
2.1 Using the auto keyword
To have different return type according to the specializations or to force compile-time checks of the method definitions, you can use the auto keyword as follows:
#include <iostream> template <typename IMPL> class Crtp { public: const IMPL& impl() const { return static_cast<const IMPL&>(*this); }; auto size() const // auto { return impl().size(); }; }; class A1 : public Crtp<A1> { public: size_t size() const { return 10; }; }; class A2 : public Crtp<A2> { public: // size_t size() const <- oppss, but now you have a // compile-time error // { // return 20; // }; };
Any attempt to compile the code causes now a compile-time error:
In instantiation of ‘auto Crtp<IMPL>::size() const [with IMPL = A2]’: error: use of ‘auto Crtp<IMPL>::size() const [with IMPL = A2]’ before deduction of ‘auto’...
This is a good thing as errors are detected earlier.
IMHO a disadvantage is that it hides the returned types of the static interface.
auto Crtp<IMPL>::size() const // what is the returned type?
2.2 Using decltype(auto)
This is a general remark. If you want to return a value, use auto, if you want to return a reference, use auto&. However in some rare cases you want some specializations to return a value and some other specializations to return a reference. In that case you must use decltype(auto) as follows:
#include <iostream> template <typename IMPL> class Crtp { public: const IMPL& impl() const { return static_cast<const IMPL&>(*this); }; decltype(auto) size() const // decltype(auto) { return impl().size(); }; }; class A1 : public Crtp<A1> { public: size_t size() const // a value { return 10; }; }; class A2 : public Crtp<A2> { public: size_t& size() const // a reference { static size_t s = 20; return s; }; }; template <typename IMPL> void foo(const Crtp<IMPL>& v) { std::cout << "\n" << v.size(); } int main() { A1 a1; // a1.size() = 30; compile time error: // lvalue required as // left operand of assignment foo(a1); A2 a2; foo(a2); a2.size() = 30; // ok foo(a2); }
10 20 30
2.3 Another way: using a type_traits
There is an alternative to auto. Actually, this was the historical approach before C++11.
Despite it requires some boilerplate code I prefer it as it makes the interface more readable.
For pedagogical purpose let’s try this and see why it does not work:
#include <iostream> template <typename IMPL> class Crtp { public: const IMPL& impl() const { return static_cast<const IMPL&>(*this); }; typename IMPL::size_type size() const // does not work { return impl().size(); }; }; class A1 : public Crtp<A1> { public: using size_type = size_t; // here public: size_type size() const // here { return 10; }; }; int main() { A1 a1; // compile-time error }
If you try to compile the code, you will get a compile-time error:
In instantiation of ‘class Crtp<A1>’: required from here error: invalid use of incomplete type ‘class A1’ typename IMPL::size_type size() const // compile-time error
The reason is when instantiating Crtp<A1> in
class A1 : public Crtp<A1> { ... }
the definition of A1 is not available (circular definition).
The solution is to use an extra independent Crtp_Type_Traits class. The working solution is as follows:
#include <iostream> #include <type_traits> template <typename IMPL> struct Crtp_Type_Traits; // here template <typename IMPL> class Crtp { public: using exact_type = IMPL; // here using trait_type = Crtp_Type_Traits<exact_type>; // here using size_type = typename trait_type::size_type; // here public: const IMPL& impl() const { return static_cast<const IMPL&>(*this); }; size_type size() const // a more readable interface { return impl().size(); }; }; class A1; template <> struct Crtp_Type_Traits<A1> // some boilerplate code { using size_type = size_t; }; class A1 : public Crtp<A1> { public: using base_type = Crtp<A1>; // here using size_type = typename base_type::size_type; // here public: size_type size() const { return 10; }; }; class A2; template <> struct Crtp_Type_Traits<A2> // some boilerplate code { using size_type = std::integral_constant<size_t, 20>; }; class A2 : public Crtp<A2> { public: using base_type = Crtp<A2>; // here using size_type = typename base_type::size_type; // here public: size_type size() const { return {}; }; }; template <typename T> struct is_integral_constant : std::false_type { }; template <typename T, T t> struct is_integral_constant<std::integral_constant<T, t>> : std::true_type { }; template <typename IMPL> void foo(const Crtp<IMPL>& v) { std::cout << "\n" << v.size() << " has static size " << std::boolalpha << is_integral_constant<decltype(v.size())>::value; } int main() { A1 a1; foo(a1); A2 a2; foo(a2); }
10 has static size false 20 has static size true
For ease of comparison here is the version using auto. It is shorter but (IMHO) less convenient to manipulate/reuse returned type information.
#include <iostream> #include <type_traits> template <typename IMPL> class Crtp { public: const IMPL& impl() const { return static_cast<const IMPL&>(*this); }; auto size() const { return impl().size(); }; }; class A1 : public Crtp<A1> { public: size_t size() const { return 10; }; }; class A2 : public Crtp<A2> { public: std::integral_constant<size_t, 20> size() const { return {}; }; }; template <typename T> struct is_integral_constant : std::false_type { }; template <typename T, T t> struct is_integral_constant<std::integral_constant<T, t>> : std::true_type { }; template <typename IMPL> void foo(const Crtp<IMPL>& v) { std::cout << "\n" << v.size() << " has static size " << std::boolalpha << is_integral_constant<decltype(v.size())>::value; } int main() { A1 a1; foo(a1); A2 a2; foo(a2); }
3 More levels in the inheritance hierarchy
Here is a less commonly explained extension of CRTP: the possibility to define a deeper hierarchy.
Here is a first, but not satisfactory, attempt:
#include <iostream> template <typename IMPL> class Crtp { public: const IMPL& impl() const { return static_cast<const IMPL&>(*this); }; size_t size() const { return impl().size(); }; }; template <typename IMPL> class A : public Crtp<IMPL> // here { public: size_t size() const { return 10; }; }; class B : public A<B> // here { public: size_t size() const { return 20; }; }; template <typename IMPL> void foo(const Crtp<IMPL>& v) { std::cout << "\n" << v.size(); } int main() { B b; foo(b); }
20
The code prints 20 as expected, but the problem now is that we cannot use the class A<> anymore:
int main() { A<> a; // how to use A<> alone? (this does not compile) B b; // ok }
Here is the right way to do things:
#include <iostream> struct Crtp_Final_Impl; // here template <typename IMPL> class Crtp { public: const IMPL& impl() const { return static_cast<const IMPL&>(*this); }; size_t size() const { return impl().size(); }; }; template <typename IMPL = Crtp_Final_Impl> class A : // --> here is the magic <-- public std::conditional< std::is_same_v<IMPL, Crtp_Final_Impl>, Crtp<A<>>, Crtp<IMPL>>::type { public: size_t size() const { return 10; }; }; class B : public A<B> // the final class does not need // special considerations, A<B> is ok { public: size_t size() const { return 20; }; }; template <typename IMPL> void foo(const Crtp<IMPL>& v) { std::cout << "\n" << v.size(); } int main() { A<> a; foo(a); B b; foo(b); }
10 20
The role of
std::conditional_t<std::is_same_v<IMPL, Crtp_Final_Impl>, Crtp<A<>>, Crtp<IMPL>>
is to define the right inherited type, namely
Crtp<A<>>whenA<> a;is instantiated, in that caseIMPL = Crtp_Final_Impl,Crtp<IMPL>otherwise. For the caseB b;this reduces toIMPL = B.
3.1 Less boilerplate code with the Crtp_Find_Impl helper
You can create more deeper hierarchy and automatize the type deduction as follows:
#include <iostream> struct Crtp_Final_Impl; template <template <typename> class THIS, typename IMPL, template <typename> class SUPERCLASS> using Crtp_Find_Impl = SUPERCLASS<std::conditional_t< std::is_same_v<IMPL, Crtp_Final_Impl>, THIS<Crtp_Final_Impl>, IMPL>>; template <typename IMPL> class Crtp { public: const IMPL& impl() const { return static_cast<const IMPL&>(*this); }; size_t size() const { return impl().size(); }; }; template <typename IMPL = Crtp_Final_Impl> class A : public Crtp_Find_Impl<A, IMPL, Crtp> { public: size_t size() const { return 10; }; }; template <typename IMPL = Crtp_Final_Impl> class B : public Crtp_Find_Impl<B, IMPL, A> { public: size_t size() const { return 20; }; }; class C : public B<C> { public: size_t size() const { return 30; }; }; template <typename IMPL> void foo(const Crtp<IMPL>& v) { std::cout << "\n" << v.size(); } int main() { A<> a; foo(a); B<> b; foo(b); C c; foo(c); }
10 20 30
3.2 A problem remains, mimicking abstract interfaces
There is still another problem. Please note this problem cannot happen if you only have one level in the hierarchy, as in the Classical case.
For comparison purpose here is a code using dynamic interfaces:
#include <iostream> class Crtp { public: virtual size_t size() const = 0; // a dynamic interface // (no crtp anymore!) }; class A : public Crtp { public: size_t size() const override // a dynamic interface // (no crtp anymore!) { return 10; }; }; class B : public A { public: size_t size() const override // a dynamic interface // (no crtp anymore!) { return 20; }; }; class C final : public B { public: size_t size() const override // a dynamic interface // (no crtp anymore!) { return 30; }; }; void foo(const Crtp& v) { std::cout << "\nfoo: " << v.size(); } void foo_a(const A& v) { std::cout << "\nfoo_a: " << v.size(); } int main() { A a; foo(a); foo_a(a); B b; foo(b); foo_a(b); C c; foo(c); foo_a(c); }
foo: 10 foo_a: 10 foo: 20 foo_a: 20 foo: 30 foo_a: 30
We see that, as expected, the foo() and foo_a() results are identical.
However if we reuse the code of the Less boilerplate code with the Crtp_Find_Impl helper section:
template <typename IMPL> void foo(const Crtp<IMPL>& v) { std::cout << "\nfoo " << v.size(); } template <typename IMPL> void foo_a(const A<IMPL>& v) { std::cout << "\nfoo_a: " << v.size(); } int main() { A<> a; foo(a); foo_a(a); B<> b; foo(b); foo_a(b); C c; foo(c); foo_a(c); }
you will get:
foo: 10 foo_a: 10 foo: 20 foo_a: 10 // problem, must get 20 foo: 30 foo_a: 10 // problem, must get 30
The origin of the problem is that when you call A::size() in foo_a() you shortcut the CRTP mechanism defined in:
size_t Crtp<IMPL>::size() const { return impl().size(); };
A solution is to use something similar to the Non-virtual_Interface_Pattern:
#include <iostream> struct Crtp_Final_Impl; template <template <typename> class THIS, typename IMPL, template <typename> class SUPERCLASS> using Crtp_Find_Impl = SUPERCLASS<std::conditional_t< std::is_same_v<IMPL, Crtp_Final_Impl>, THIS<Crtp_Final_Impl>, IMPL>>; template <typename IMPL> class Crtp { public: using exact_type = IMPL; // used to get type needed by // friend declarations public: const IMPL& impl() const { return static_cast<const IMPL&>(*this); }; size_t size() const { return impl() .impl_size(); // do not call size() but a protected // implementation impl_size() }; }; template <typename IMPL = Crtp_Final_Impl> class A : public Crtp_Find_Impl<A, IMPL, Crtp> { public: // boilerplate code required // by the necessary friend declaration using base_type = Crtp_Find_Impl<A, IMPL, Crtp>; using exact_type = typename base_type::exact_type; friend Crtp<exact_type>; protected: // the protected implementation size_t impl_size() const { return 10; }; }; template <typename IMPL = Crtp_Final_Impl> class B : public Crtp_Find_Impl<B, IMPL, A> { public: // ditto using base_type = Crtp_Find_Impl<B, IMPL, A>; using exact_type = typename base_type::exact_type; friend Crtp<exact_type>; protected: // ditto size_t impl_size() const { return 20; }; }; class C : public B<C> { public: // ditto using base_type = B<C>; using exact_type = typename base_type::exact_type; friend Crtp<exact_type>; protected: // ditto size_t impl_size() const { return 30; }; }; template <typename IMPL> void foo(const Crtp<IMPL>& v) { std::cout << "\nfoo: " << v.size(); } template <typename IMPL> void foo_a(const A<IMPL>& v) { std::cout << "\nfoo_a: " << v.size(); } int main() { A<> a; foo(a); foo_a(a); B<> b; foo(b); foo_a(b); C c; foo(c); foo_a(c); }
foo: 10 foo_a: 10 foo: 20 foo_a: 20 foo: 30 foo_a: 30
Now the problem is fixed and the result is identical to the abstract interface behavior.
3.3 A final trick, compile-time check for interface
This trick is not really important, it is more for completeness that I present it. It allows you to check at compile-time if an object implements a given CRTP interface. This can be useful for SFINAE for instance.
Here is the code:
template <template <typename> class CRTP_INTERFACE, typename OBJ, typename ENABLE = void> struct is_crtp_interface_of : std::false_type { }; template <template <typename> class CRTP_INTERFACE, typename OBJ> struct is_crtp_interface_of< CRTP_INTERFACE, OBJ, std::enable_if_t< std::is_same_v<CRTP_INTERFACE<Crtp_Final_Impl>, OBJ> || std::is_base_of_v< CRTP_INTERFACE<typename OBJ::exact_type>, OBJ>>> : std::true_type { };
A usage demo reusing the A problem remains, mimicking abstract interfaces code is as follows:
int main() { static_assert(is_crtp_interface_of<B, B<>>::value); static_assert(is_crtp_interface_of<A, C>::value); static_assert(!is_crtp_interface_of<B, A<>>::value); static_assert(!is_crtp_interface_of<A, int>::value); }
4 Putting everything together
A this point we have everything we need to properly use CRTP.
A sample code grouping all the tips we discussed is as follows:
#include <iostream> #include <type_traits> #include <iostream> struct Crtp_Final_Impl; template <template <typename> class THIS, typename IMPL, template <typename> class SUPERCLASS> using Crtp_Find_Impl = SUPERCLASS<std::conditional_t< std::is_same_v<IMPL, Crtp_Final_Impl>, THIS<Crtp_Final_Impl>, IMPL>>; template <typename IMPL> struct Crtp_Type_Traits; template <typename IMPL> class Crtp { public: using exact_type = IMPL; using size_type = typename Crtp_Type_Traits<exact_type>::size_type; public: const exact_type& impl() const { return static_cast<const exact_type&>(*this); }; exact_type& impl() { return static_cast<exact_type&>(*this); }; size_type size() const { return impl().impl_size(); }; }; template <typename IMPL> class A1; template <typename IMPL> struct Crtp_Type_Traits<A1<IMPL>> { using size_type = size_t; }; template <typename IMPL = Crtp_Final_Impl> class A1 : public Crtp_Find_Impl<A1, IMPL, Crtp> { public: using base_type = Crtp_Find_Impl<A1, IMPL, Crtp>; using exact_type = typename base_type::exact_type; friend Crtp<exact_type>; using size_type = typename base_type::size_type; protected: size_t _n; public: A1(size_t n) : _n(n) { } protected: size_type impl_size() const { return _n; }; }; class A2; template <> struct Crtp_Type_Traits<A2> : Crtp_Type_Traits<A1<A2>> { }; class A2 : public A1<A2> { public: using base_type = A1<A2>; using exact_type = typename base_type::exact_type; friend Crtp<exact_type>; using size_type = typename base_type::size_type; public: A2(size_t n) : base_type(n){}; }; template <size_t N> class B1; template <size_t N> struct Crtp_Type_Traits<B1<N>> { using size_type = std::integral_constant<size_t, N>; }; template <size_t N> class B1 : public Crtp<B1<N>> { public: using base_type = Crtp<B1<N>>; using exact_type = typename base_type::exact_type; friend Crtp<exact_type>; using size_type = typename base_type::size_type; protected: size_type impl_size() const { return {}; }; }; int main() { A1<> a1(1); A2 a2(2); B1<3> b1; }
5 Keep light classes, use functions
Now let’s discuss a little bit on how to use CRTP classes.
I personally prefer to keep the number of class methods as low as possible and use functions.
With functions you are not intrusive and have complex dispatches.
Using the Putting everything together code, here is some illustrations in the next subsections.
5.1 Construction by functions returning the right type
Functions can greatly facilitate the construction of the objects. One big advantage is that you can automatically return the best type according to the input argument types.
A2 create(size_t n) { return A2(n); }; template <size_t N> B1<N> create(std::integral_constant<size_t, N>) { return {}; }; int main() { auto dynamic_size_obj = create(10); auto static_size_obj = create(std::integral_constant<size_t, 10>()); }
5.2 Generic functions
With functions you can easily create generic code
template <typename IMPL, typename... TAIL_IMPL> bool same_size(const Crtp<IMPL>& obj, const Crtp<TAIL_IMPL>&... tail_obj) { return ((obj.size() == tail_obj.size()) && ...); } int main() { auto dynamic_size_obj = create(10); auto static_size_obj = create(std::integral_constant<size_t, 10>()); assert(same_size(dynamic_size_obj, static_size_obj, dynamic_size_obj)); }
5.3 Powerful dispatches
Most importantly you can have complex dispatches
namespace Details { template <typename IMPL1, typename IMPL2> void do_something(Crtp<IMPL1>&, Crtp<IMPL2>&) { std::cout << "\n default implementation"; } template <size_t N> void do_something(B1<N>&, B1<N>&) { std::cout << "\n static size specialization"; } } // namespace Details template <typename IMPL1, typename IMPL2> void do_something(Crtp<IMPL1>& o1, Crtp<IMPL2>& o2) { Details::do_something(o1.impl(), o2.impl()); } int main() { A1<> a1(1); A2 a2(2); B1<3> b1; do_something(b1, b1); // "static size specialization" do_something(b1, a1); // "default implementation" }
6 It’s over for now
I hope this post was instructive and that I will have opportunities to post more complex usages. I personally use CRTP for some linear algebra and automatic differentiation codes.
A Meson starter script for C++ projects
What is it?
A simple script to create a Meson C++ project skeleton with Doxygen and gtest configured. The whole script can be found in this GitHub repo.
Usage:
./create_project --help
Create directory --help
Usage: mkdir [OPTION]... DIRECTORY...
Create the DIRECTORY(ies), if they do not already exist.
Mandatory arguments to long options are mandatory for short options too.
-m, --mode=MODE set file mode (as in chmod), not a=rwx - umask
-p, --parents no error if existing, make parent directories as needed
-v, --verbose print a message for each created directory
-Z set SELinux security context of each created directory
to the default type
--context[=CTX] like -Z, or if CTX is specified then set the SELinux
or SMACK security context to CTX
--help display this help and exit
--version output version information and exit
or available locally via: info '(coreutils) mkdir invocation'
Create file --help/meson.build
./create_project: line 48: --help/meson.build: No such file or directory
Detailed example
Project creation
To create a skeleton project in /tmp/MyProject, run
./create_project /tmp/MyProject
Create directory /tmp/MyProject Create file /tmp/MyProject/meson.build Create directory /tmp/MyProject/src Create file /tmp/MyProject/src/meson.build Create directory /tmp/MyProject/src/MyProject Create file /tmp/MyProject/src/MyProject/meson.build Create file /tmp/MyProject/src/MyProject/hello.hpp Create file /tmp/MyProject/src/MyProject/hello.cpp Create directory /tmp/MyProject/subprojects Create file /tmp/MyProject/subprojects/gtest.wrap Create directory /tmp/MyProject/test Create file /tmp/MyProject/test/meson.build Create file /tmp/MyProject/test/hello.cpp Create directory /tmp/MyProject/bin Create file /tmp/MyProject/bin/meson.build Create file /tmp/MyProject/bin/hello.cpp Create directory /tmp/MyProject/sandbox Create file /tmp/MyProject/sandbox/meson.build Create file /tmp/MyProject/sandbox/hello.cpp Create directory /tmp/MyProject/examples Create file /tmp/MyProject/examples/meson.build Create file /tmp/MyProject/examples/hello.cpp Create directory /tmp/MyProject/doc Create directory /tmp/MyProject/doc/figures Create file /tmp/MyProject/doc/meson.build Create file /tmp/MyProject/doc/Doxyfile.in Create file /tmp/MyProject/doc/bibliography.bib
You get a basic Meson project with the following directory hierarchy:
+ meson.build <- root meson file + src/ + MyProject/ <- your "MyProject" library (installed, scanned by Doxygen) + hello.cpp + hello.hpp + meson.build + meson.build + bin/ <- binaries (installed, scanned by Doxygen) + meson.build + hello.cpp + sandbox/ <- "sandbox" binaries (not installed, not scanned by Doxygen) + meson.build + hello.cpp + examples/ <- "examples" binaries (not installed, Doxygen examples dir) + meson.build + hello.cpp + test/ <- unit tests (not installed) + meson.build + hello.cpp + doc/ + meson.build + bibliography.bib <- doxygen + bibliography file (installed) + Doxyfile.in + subprojects/ <- subprojects, here gtest (not installed) + gtest.wrap
Build dir creation
To create your project build directory with /tmp/InstallDirectory as installation directory run:
cd /tmp/MyProject
meson build --prefix=/tmp/InstallDirectory
The Meson build system Version: 0.49.2 Source dir: /tmp/MyProject Build dir: /tmp/MyProject/build Build type: native build Project name: MyProject Project version: 0.0.1 Native C++ compiler: ccache c++ (gcc 8.3.0 "c++ (Debian 8.3.0-6) 8.3.0") Build machine cpu family: x86_64 Build machine cpu: x86_64 Downloading gtest source from https://github.com/google/googletest/archive/release-1.8.0.zip Downloading file of unknown size. Downloading gtest patch from https://wrapdb.mesonbuild.com/v1/projects/gtest/1.8.0/5/get_zip Download size: 2084 Downloading: .......... | |Executing subproject gtest | |Project name: gtest |Project version: 1.8.0 |Native C++ compiler: ccache c++ (gcc 8.3.0 "c++ (Debian 8.3.0-6) 8.3.0") |Dependency threads found: YES |Dependency threads found: YES (cached) |Dependency threads found: YES (cached) |Dependency threads found: YES (cached) |Build targets in project: 0 | |Subproject gtest finished. Program doxygen found: YES (/usr/bin/doxygen) Program dot found: YES (/usr/bin/dot) Configuring Doxyfile using configuration Build targets in project: 6 Found ninja-1.8.2 at /usr/bin/ninja
Project compilation
To compile your project (with its doxygen documentation) type:
cd build
ninja
[1/14] Compiling C++ object 'src/MyProject/63d4f26@@MyProject@sha/hello.cpp.o'. [2/14] Compiling C++ object 'sandbox/b7ad567@@hello@exe/hello.cpp.o'. [3/14] Compiling C++ object 'test/9f86d08@@hello_exe@exe/.._subprojects_googletest-release-1.8.0_googletest_src_gtest_main.cc.o'. [4/14] Compiling C++ object 'test/9f86d08@@hello_exe@exe/hello.cpp.o'. [5/14] Compiling C++ object 'bin/51a1f05@@hello@exe/hello.cpp.o'. [6/14] Compiling C++ object 'test/9f86d08@@hello_exe@exe/.._subprojects_googletest-release-1.8.0_googletest_src_gtest-all.cc.o'. [7/14] Compiling C++ object 'examples/c590b3c@@hello@exe/hello.cpp.o'. [8/14] Linking target src/MyProject/libMyProject.so. [9/14] Generating doc with a custom command. warning: ignoring unknown tag `y' at line 2277, file doc/Doxyfile Searching for include files... Searching for files in directory /tmp/MyProject/src/MyProject Searching for files in directory /tmp/MyProject/bin Searching for example files... Searching for files in directory /tmp/MyProject/examples Searching for images... Searching for files in directory /tmp/MyProject/doc/figures Searching for dot files... Searching for msc files... Searching for dia files... Searching for files to exclude Searching INPUT for files to process... Searching for files in directory /tmp/MyProject/src/MyProject Searching for files in directory /tmp/MyProject/bin Reading and parsing tag files Parsing files Preprocessing /tmp/MyProject/src/MyProject/hello.cpp... Parsing file /tmp/MyProject/src/MyProject/hello.cpp... Preprocessing /tmp/MyProject/src/MyProject/hello.hpp... Parsing file /tmp/MyProject/src/MyProject/hello.hpp... Preprocessing /tmp/MyProject/bin/hello.cpp... Parsing file /tmp/MyProject/bin/hello.cpp... Building group list... Building directory list... Building namespace list... Building file list... Building class list... Associating documentation with classes... Computing nesting relations for classes... Building example list... Searching for enumerations... Searching for documented typedefs... Searching for members imported via using declarations... Searching for included using directives... Searching for documented variables... Building interface member list... Building member list... Searching for friends... Searching for documented defines... Computing class inheritance relations... Computing class usage relations... Flushing cached template relations that have become invalid... Computing class relations... Add enum values to enums... Searching for member function documentation... Creating members for template instances... Building page list... Search for main page... Computing page relations... Determining the scope of groups... Sorting lists... Freeing entry tree Determining which enums are documented Computing member relations... Building full member lists recursively... Adding members to member groups. Computing member references... Inheriting documentation... Generating disk names... Adding source references... Adding xrefitems... Sorting member lists... Generating citations page... Counting data structures... Resolving user defined references... Finding anchors and sections in the documentation... Transferring function references... Combining using relations... Adding members to index pages... Generating style sheet... Generating search indices... Generating example documentation... Generating file sources... Generating code for file hello.hpp... Generating file documentation... Generating docs for file hello.cpp... Generating docs for file hello.hpp... Generating page documentation... Generating docs for page citelist... Generating group documentation... Generating class documentation... Generating namespace index... Generating graph info page... Generating directory documentation... Generating index page... Generating page index... Generating module index... Generating namespace index... Generating namespace member index... Generating annotated compound index... Generating alphabetical compound index... Generating hierarchical class index... Generating member index... Generating file index... Generating file member index... Generating example index... finalizing index lists... writing tag file... Running dot... Running dot for graph 1/1 lookup cache used 5/65536 hits=10 misses=5 finished... [10/14] Generating symbol file 'src/MyProject/63d4f26@@MyProject@sha/libMyProject.so.symbols'. [11/14] Linking target bin/hello. [12/14] Linking target sandbox/hello. [13/14] Linking target examples/hello. [14/14] Linking target test/hello_exe.
Note: You can visualize the generated documentation with:
chromium doc/html/index.html
Running unit tests
To run unit tests, run:
ninja test
[1/2] Generating doc with a custom command. warning: ignoring unknown tag `y' at line 2277, file doc/Doxyfile Searching for include files... Searching for files in directory /tmp/MyProject/src/MyProject Searching for files in directory /tmp/MyProject/bin Searching for example files... Searching for files in directory /tmp/MyProject/examples Searching for images... Searching for files in directory /tmp/MyProject/doc/figures Searching for dot files... Searching for msc files... Searching for dia files... Searching for files to exclude Searching INPUT for files to process... Searching for files in directory /tmp/MyProject/src/MyProject Searching for files in directory /tmp/MyProject/bin Reading and parsing tag files Parsing files Preprocessing /tmp/MyProject/src/MyProject/hello.cpp... Parsing file /tmp/MyProject/src/MyProject/hello.cpp... Preprocessing /tmp/MyProject/src/MyProject/hello.hpp... Parsing file /tmp/MyProject/src/MyProject/hello.hpp... Preprocessing /tmp/MyProject/bin/hello.cpp... Parsing file /tmp/MyProject/bin/hello.cpp... Building group list... Building directory list... Building namespace list... Building file list... Building class list... Associating documentation with classes... Computing nesting relations for classes... Building example list... Searching for enumerations... Searching for documented typedefs... Searching for members imported via using declarations... Searching for included using directives... Searching for documented variables... Building interface member list... Building member list... Searching for friends... Searching for documented defines... Computing class inheritance relations... Computing class usage relations... Flushing cached template relations that have become invalid... Computing class relations... Add enum values to enums... Searching for member function documentation... Creating members for template instances... Building page list... Search for main page... Computing page relations... Determining the scope of groups... Sorting lists... Freeing entry tree Determining which enums are documented Computing member relations... Building full member lists recursively... Adding members to member groups. Computing member references... Inheriting documentation... Generating disk names... Adding source references... Adding xrefitems... Sorting member lists... Generating citations page... Counting data structures... Resolving user defined references... Finding anchors and sections in the documentation... Transferring function references... Combining using relations... Adding members to index pages... Generating style sheet... Generating search indices... Generating example documentation... Generating file sources... Generating code for file hello.hpp... Generating file documentation... Generating docs for file hello.cpp... Generating docs for file hello.hpp... Generating page documentation... Generating docs for page citelist... Generating group documentation... Generating class documentation... Generating namespace index... Generating graph info page... Generating directory documentation... Generating index page... Generating page index... Generating module index... Generating namespace index... Generating namespace member index... Generating annotated compound index... Generating alphabetical compound index... Generating hierarchical class index... Generating member index... Generating file index... Generating file member index... Generating example index... finalizing index lists... writing tag file... Running dot... lookup cache used 5/65536 hits=10 misses=5 finished... [1/2] Running all tests. 1/1 hello_test OK 0.01 s Ok: 1 Expected Fail: 0 Fail: 0 Unexpected Pass: 0 Skipped: 0 Timeout: 0 Full log written to /tmp/MyProject/build/meson-logs/testlog.txt
Project installation
Finally, if you want to install your project, run:
ninja install
[1/2] Generating doc with a custom command. warning: ignoring unknown tag `y' at line 2277, file doc/Doxyfile Searching for include files... Searching for files in directory /tmp/MyProject/src/MyProject Searching for files in directory /tmp/MyProject/bin Searching for example files... Searching for files in directory /tmp/MyProject/examples Searching for images... Searching for files in directory /tmp/MyProject/doc/figures Searching for dot files... Searching for msc files... Searching for dia files... Searching for files to exclude Searching INPUT for files to process... Searching for files in directory /tmp/MyProject/src/MyProject Searching for files in directory /tmp/MyProject/bin Reading and parsing tag files Parsing files Preprocessing /tmp/MyProject/src/MyProject/hello.cpp... Parsing file /tmp/MyProject/src/MyProject/hello.cpp... Preprocessing /tmp/MyProject/src/MyProject/hello.hpp... Parsing file /tmp/MyProject/src/MyProject/hello.hpp... Preprocessing /tmp/MyProject/bin/hello.cpp... Parsing file /tmp/MyProject/bin/hello.cpp... Building group list... Building directory list... Building namespace list... Building file list... Building class list... Associating documentation with classes... Computing nesting relations for classes... Building example list... Searching for enumerations... Searching for documented typedefs... Searching for members imported via using declarations... Searching for included using directives... Searching for documented variables... Building interface member list... Building member list... Searching for friends... Searching for documented defines... Computing class inheritance relations... Computing class usage relations... Flushing cached template relations that have become invalid... Computing class relations... Add enum values to enums... Searching for member function documentation... Creating members for template instances... Building page list... Search for main page... Computing page relations... Determining the scope of groups... Sorting lists... Freeing entry tree Determining which enums are documented Computing member relations... Building full member lists recursively... Adding members to member groups. Computing member references... Inheriting documentation... Generating disk names... Adding source references... Adding xrefitems... Sorting member lists... Generating citations page... Counting data structures... Resolving user defined references... Finding anchors and sections in the documentation... Transferring function references... Combining using relations... Adding members to index pages... Generating style sheet... Generating search indices... Generating example documentation... Generating file sources... Generating code for file hello.hpp... Generating file documentation... Generating docs for file hello.cpp... Generating docs for file hello.hpp... Generating page documentation... Generating docs for page citelist... Generating group documentation... Generating class documentation... Generating namespace index... Generating graph info page... Generating directory documentation... Generating index page... Generating page index... Generating module index... Generating namespace index... Generating namespace member index... Generating annotated compound index... Generating alphabetical compound index... Generating hierarchical class index... Generating member index... Generating file index... Generating file member index... Generating example index... finalizing index lists... writing tag file... Running dot... lookup cache used 5/65536 hits=10 misses=5 finished... [1/2] Installing files. Installing src/MyProject/libMyProject.so to /tmp/InstallDirectory/lib/x86_64-linux-gnu Installing bin/hello to /tmp/InstallDirectory/bin Installing /tmp/MyProject/build/doc/html/a00005.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/tab_h.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/sync_on.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/graph_legend.md5 to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/dir_68267d1309a1af8e8297ef4c3efbcdba.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/pages.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/jquery.js to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/menu.js to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/folderclosed.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/dir_12739b51ecd9f955b0759ea118dd308c.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/nav_f.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/tab_s.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/splitbar.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/graph_legend.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/closed.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/bdwn.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/a00015.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/tabs.css to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/doxygen.css to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/index.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/tab_a.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/open.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/menudata.js to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/graph_legend.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/nav_h.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/a00012.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/bc_s.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/doxygen.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/nav_g.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/dynsections.js to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/dir_2ea30aa2956a8db99dd22aa5e597f384.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/files.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/sync_off.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/folderopen.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/a00005_source.html to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/tab_b.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/doc.png to /tmp/InstallDirectory/share/doc/html Installing /tmp/MyProject/build/doc/html/search/files_1.js to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/all_1.js to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/search_r.png to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/all_0.js to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/close.png to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/search.js to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/search_l.png to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/files_0.html to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/searchdata.js to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/nomatches.html to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/pages_0.html to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/all_0.html to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/files_1.html to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/mag_sel.png to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/search.css to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/pages_0.js to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/all_1.html to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/search_m.png to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/build/doc/html/search/files_0.js to /tmp/InstallDirectory/share/doc/html/search Installing /tmp/MyProject/src/MyProject/hello.hpp to /tmp/InstallDirectory/include/MyProject
Further information
Project file listings
The MyProject library
cat /tmp/MyProject/src/MyProject/hello.hpp
/// @file
/// @brief Hello lib file
include <string>
namespace MyProject {
/// @brief A function
///
/// A bibtex reference example @cite pakkanen_meson_build_system
///
/// Usage example @include hello.cpp
std::string hello();
}
cat /tmp/MyProject/src/MyProject/hello.cpp
include "hello.hpp"
namespace MyProject {
std::string hello() {
return "Hello MyProject";
};
}
The project unit tests
cat /tmp/MyProject/test/hello.cpp
include "MyProject/hello.hpp"
include <gtest/gtest.h>
using namespace MyProject;
TEST(Hello, demo)
{
ASSERT_EQ(hello(),"Hello MyProject");
}
The bin/hello.cpp file
cat /tmp/MyProject/bin/hello.cpp
/// @file
/// @brief Hello executable file
include "MyProject/hello.hpp"
include <iostream>
using namespace MyProject;
int main()
{
std::cout << hello() << " from bin/ " << std::endl;
}
The sandbox/hello.cpp file
cat /tmp/MyProject/sandbox/hello.cpp
/// @file
/// @brief Hello example file
include "MyProject/hello.hpp"
include <iostream>
using namespace MyProject;
int main()
{
std::cout << hello() << " from sandbox/ " << std::endl;
}
The examples/hello.cpp file
cat /tmp/MyProject/examples/hello.cpp
/// @file
/// @brief Hello example file
include "MyProject/hello.hpp"
include <iostream>
using namespace MyProject;
int main()
{
std::cout << hello() << " from examples/ " << std::endl;
}
More about Meson
Some Meson official site links:
Wolfram LibraryLink + Meson
I just released a demo/tutorial about Wolfram Library Link + C++ integration with the Meson build system. The GitHub repository is here.
Table of Contents
1 What is it?
The goal is to use the Meson build system to define an easy to use and
portable solution for Mathematica (MMA) Wolfram LibraryLink
developments. This GitHub repository is a small illustrative example.
The important files are:
+ meson.build + mma/ + algorithm1_mma.cpp + config.wls + meson.build + myLib_mma.cpp + myLib.wl + src/ + meson.build + myLib/ + algorithm1.cpp + algorithm1.hpp + meson.build
2 How to use it?
2.1 Compile and install the package
git clone git@github.com:vincent-picaud/mma_meson_demo.git cd mma_meson_demo meson build --prefix=$(pwd)/install_dir cd build ninja install
you will get something like:
picaud@hostname:~/GitHub/mma_meson_demo$ meson build --prefix=$(pwd)/install_dir The Meson build system Version: 0.49.2 Source dir: /home/picaud/GitHub/mma_meson_demo Build dir: /home/picaud/GitHub/mma_meson_demo/build Build type: native build Project name: Meson_MMA_Demo Project version: undefined Native C++ compiler: ccache c++ (gcc 8.3.0 "c++ (Debian 8.3.0-2) 8.3.0") Build machine cpu family: x86_64 Build machine cpu: x86_64 Program wolframscript found: YES (/usr/bin/wolframscript) Message: MMA library installation directory: /home/picaud/.Mathematica/SystemFiles/LibraryResources/Linux-x86-64 Message: MMA package installation directory: /home/picaud/.Mathematica/Applications Build targets in project: 2 Found ninja-1.8.2 at /usr/bin/ninja picaud@hostname:~/GitHub/mma_meson_demo$ cd build/ picaud@hostname:~/GitHub/mma_meson_demo/build$ ninja install [5/6] Installing files. Installing src/myLib/libmyLib.so to /home/picaud/GitHub/mma_meson_demo/install_dir/lib/x86_64-linux-gnu Installing mma/libmyLibMMA.so to /home/picaud/.Mathematica/SystemFiles/LibraryResources/Linux-x86-64 Installing /home/picaud/GitHub/mma_meson_demo/src/myLib/algorithm1.hpp to /home/picaud/GitHub/mma_meson_demo/install_dir/include/myLib Installing /home/picaud/GitHub/mma_meson_demo/mma/myLib.wl to /home/picaud/.Mathematica/Applications
Remark: [role of –prefix]
This demo will generate two dynamic libraries:
libmyLib.soa regular C++ library (totally independent of MMA)libmylibMMA.soa dynamic library that uses MMA’s LibraryLink to wrap functions contained inlibmyLib.so.The
libmyLib.soinstall directory is defined by the--prefixflag, with--prefix=$(pwd)/install_dirthis is/home/picaud/GitHub/mma_meson_demo/install_dir/lib/x86_64-linux-gnuin my case.
By default
libmylibMMA.sois installed in the user directory$HOME/.Mathematica/SystemFiles/LibraryResources/Linux-x86-64in my case.
The important point to understand is that
libmylibMMA.soneeds to know
where thelibmyLib.solibrary is installed. We provide this
information by embedding it into thelibmylibMMA.sorpath
section. This section is filled by Meson at installation time. By
consequence to make it works we must define this installation path.If the
--prefixvariable is not defined everything works equally well
except that you will need extra permissions to do a default system
install (invoked when you typeninja install).
2.2 Mathematica side
To test it works, type the following under Mathematica:
<< myLib` ?"myLib`*" data = {2, 1, 3, 5, 6, 6, 4, 3 }; algorithm1[100.,data]
it should print:
algorithm1[α_Real,vector_?VectorQ] returns α.vector
{200., 100., 300., 500., 600., 600., 400., 300.}
Remark: [
myLib`Private`unload[]]If you recompile the
libmylib.soorlibmylibMMA.soC++ library you
must callmyLib`Private`unload[]to make MMA aware of this
modification (no need to kill the Kernel!).Obviously, for regular uses of the MMA
myLibpackage, you do not need
this function.
3 Portability
I have no access to Windows. It would we nice to have some feedback
concerning this platform:
[X]Mathematica + Linux[X]Mathematica + MacOS[ ]Mathematica + Windows ???
4 How it works?
4.1 meson.build
#---------------- # Regular Meson C++ project #---------------- project('Meson_MMA_Demo', 'cpp') subdir('src') #---------------- # MMA specific #---------------- mma_wolframscript = find_program('wolframscript') if mma_wolframscript.found() subdir('mma') else warning('Mathematica not found!') endif
The src directory contains a regular C++ meson project.
After having compiled it we test if the wolframscript executable is present, if so
we move to the mma directory.
Remark:
Even if the
wolframscriptexecutable is not present, the regularlibmylib.soC++ library
is properly compiled and installed.
4.2 ./src/meson.build
Ax explained, this is a regular C++ Meson project:
myLib_inc = include_directories('.') subdir('myLib')
4.2.1 ./src/myLib/meson.build
For this small demo we create a dynamic library libmyLib.so containing
only one file algorithm1.cpp. You can obviously add others files by
appending them to the myLib_headers and myLib_sources Meson variables.
myLib_headers = ['algorithm1.hpp'] myLib_sources = ['algorithm1.cpp'] myLib_lib = library('myLib', include_directories : myLib_inc, install : true, sources: [myLib_headers,myLib_sources]) myLib_dep = declare_dependency(include_directories : myLib_inc, link_with : myLib_lib) install_headers(myLib_headers, subdir : 'myLib')
4.3 ./mma/meson.build
This part is specific to MMA.
#---------------- # Extract MMA information #---------------- maa_config = run_command(mma_wolframscript,'-f',files('config.wls'), check: true) maa_config = maa_config.stdout().split(';') mma_include_directories = include_directories(maa_config.get(0).split(',')) mma_library_install_dir = maa_config.get(1).strip() # caveat: strip is mandatory to get mma_package_install_dir = maa_config.get(2).strip() # a correct filename message('MMA library installation directory: '+mma_library_install_dir) message('MMA package installation directory: '+mma_package_install_dir) #---------------- # myLibMMA library #---------------- myLibMMA_sources = ['myLib_mma.cpp', 'algorithm1_mma.cpp'] shared_library('myLibMMA', sources: [myLibMMA_sources], dependencies: [myLib_dep], include_directories: mma_include_directories, install: true, # libmyLibMMA.so needs to find libmyLib.so, this can be done using rpath install_rpath: join_paths(get_option('prefix'),get_option('libdir')), install_dir: mma_library_install_dir) #---------------- # MMA package #---------------- install_data('myLib.wl', install_dir: mma_package_install_dir )
In a first step we run the config.wls script to extract from MMA the
relevant information required by the Meson build process. These
information are printed in a form easily readable by Meson:
wolframscript -f config.wls
/usr/local/Wolfram/Mathematica/11.2/SystemFiles/IncludeFiles/C,/usr/local/Wolfram/Mathematica/11.2/SystemFiles/Links/MathLink/DeveloperKit/Linux-x86-64/CompilerAdditions;/home/picaud/.Mathematica/SystemFiles/LibraryResources/Linux-x86-64;/home/picaud/.Mathematica/Applications
These extracted information are stored into the mma_include_directories, mma_library_install_dir and mma_package_install_dir Meson variables.
In a second step we create the libmylibMMA.so dynamic library and also
define its rpath variable to allow it to find the installed
libmyLib.so library (see Compile and install the package).
In a third step we define where the MMA package myLib.wl will be
installed (here in the mma_package_install_dir default location).
That’s it!
4.4 config.wls
The config.wls script extracts the relevant information required by the
Meson build process.
libraryLinkIncludeDirectories={FileNameJoin[{$InstallationDirectory,"SystemFiles","IncludeFiles","C"}],
FileNameJoin[{$InstallationDirectory,"SystemFiles","Links","MathLink","DeveloperKit",$SystemID,"CompilerAdditions"}]};
libraryInstallDirectory=FileNameJoin[{$UserBaseDirectory,"SystemFiles","LibraryResources",$SystemID}];
packageInstallDirectory=FileNameJoin[{$UserBaseDirectory,"Applications"}];
(* MMA < v10.1 does not have native StringRiffle *)
stringRiffle[stringList_List,sep_String]:=TextString[stringList, ListFormat -> {"", sep, ""}];
format[s_List]:=stringRiffle[s,","]
(* stdout result in a format Meson can read *)
Print[format[libraryLinkIncludeDirectories]<>";"<>libraryInstallDirectory<>";"<>packageInstallDirectory]
4.5 Our cpp files
This is really for demo purpose as we simply compute a scalar-vector product w=α.v
4.5.1 The libmyLib.so cpp files (our c++ library)
The ./src/myLib/algorithm1.hpp file:
#pragma once #include <cstddef> namespace myLib { // For demo purpose: dest <- alpha.source void algorithm1(const double alpha,const double* source, double* dest, const size_t n); }
The ./src/myLib/algorithm1.cpp file:
#include "myLib/algorithm1.hpp" namespace myLib { void algorithm1(const double alpha, const double* source, double* dest, const size_t n) { for (size_t i = 0; i < n; i++) { dest[i] = alpha*source[i]; } } } // namespace myLib
4.5.2 The libmyLibMMA.so cpp files (our MMA wrapper)
The ./mma/myLib_mma.cpp file:
#include "WolframLibrary.h" #include "WolframSparseLibrary.h" extern "C" DLLEXPORT mint WolframLibrary_getVersion() { return WolframLibraryVersion; } extern "C" DLLEXPORT int WolframLibrary_initialize(WolframLibraryData libData) { return LIBRARY_NO_ERROR; } extern "C" DLLEXPORT void WolframLibrary_uninitialize(WolframLibraryData libData) { return; }
The ./mma/algorithm1_mma.cpp file:
#include <cassert> #include <iostream> #include "WolframLibrary.h" #include "WolframSparseLibrary.h" #include "myLib/algorithm1.hpp" //---------------- // TODO replace asserts by MMA error message extern "C" DLLEXPORT int algorithm1(WolframLibraryData libData, mint Argc, MArgument* Args, MArgument Res) { // alpha const double alpha = MArgument_getReal(Args[0]); // source vector const MTensor src_tensor = MArgument_getMTensor(Args[1]); assert(libData->MTensor_getType(src_tensor)==MType_Real); const mint src_rank = libData->MTensor_getRank(src_tensor); assert(src_rank == 1); const mint* const src_dims = libData->MTensor_getDimensions(src_tensor); const double* const src_data = libData->MTensor_getRealData(src_tensor); // dest vector MTensor dest_tensor; const mint dest_rank = src_rank; const mint dest_dims[] = { src_dims[0] }; libData->MTensor_new(MType_Real, dest_rank, dest_dims, &dest_tensor); double* const dest_data = libData->MTensor_getRealData(dest_tensor); // call our libmyLib.so function myLib::algorithm1(alpha,src_data,dest_data,src_dims[0]); MArgument_setMTensor(Res, dest_tensor); return LIBRARY_NO_ERROR; }
5 Useful references
- Doing nothing with LibraryLink certainly the place to begin with if
you do not know LibraryLink yet. - a short but instructive video about LibraryLink some tips
- Wolfram LibraryLink User Guide (official)
- https://github.com/arnoudbuzing/wolfram-librarylink-examples
Created: 2019-03-25 Mon 21:12
Managing a collection of git repositories
A small post to point out the existence of myrepos.
This tool allows you to manage a collection of git repositories.
Quoted from the official site:
You have a lot of version control repositories. Sometimes you want to update them all at once. Or push out all your local changes. You use special command lines in some repositories to implement specific workflows. Myrepos provides a mr command, which is a tool to manage all your version control repositories.
Imagine that you have several git projects ~/GitLab/project_A and ~/GitHub/project_B, etc…
Using mr is really easy. Simply register your projects:
cd mr register GitLab/project_A mr register GitHub/project_B
and that’s it!
Note: you can see your registered repos in the ~/.mrconfig configuration file.
Now you can perform operations like status, update or push for all your repositories in one command line.
mr status
mr status: /home/picaud/GitLab/project_A mr status: /home/picaud/GitHub/project_B mr status: finished (2 ok)
If you modify some files:
cd /home/picaud/GitHub/project_B echo "some text" > another_file.cpp git add another_file.cpp
then
cd mr status
will print:
mr status: /home/picaud/GitLab/project_A mr status: /home/picaud/GitHub/project_B A another_file.cpp mr status: finished (2 ok)
If you want to push everything (well, this is not a good practice):
cd mr commit -a -m "must leave the office in emergency..."
this will commits changes and push all your repos to their remotes.
Attention for the git users: the mr commit ... is equivalent to the two git operations: git commit ... followed by a git push. With mr, if you want to proceed step by step, use:
mr record -m "a commit message" mr push
A very convenient command I ritually do every starting work day is:
cd mr update
this will update (fetch + pull) all your repos in one command line!
Note:
mris available as a Debian package- is compatible with several versioning systems like
git,svn,hg… - here is the available commands (from
mrmanpage):
mr [options] checkout
mr [options] update
mr [options] status
mr [options] clean [-f]
mr [options] commit [-m "message"]
mr [options] record [-m "message"]
mr [options] fetch
mr [options] push
mr [options] diff
mr [options] log
mr [options] grep pattern
mr [options] run command [param ...]
mr [options] bootstrap src [directory]
mr [options] register [repository]
mr [options] config section ["setting=[value]" ...]
mr [options] action [params ...]
mr [options] [online|offline]
mr [options] remember action [params ...]
Generating Julia doc into Org-Mode documents
Table of Contents
1 Context
This post presents J4Org.jl a Julia package I have started to develop to include Julia doc into Org-Mode documents. My goal was to be able to code and document Julia packages without leaving Emacs and to reduce as much as possible the burden of documentation.
2 Julia code documentation
Here is a short example. Imagine that your package is:
module Foo export Point, foo import Base: norm #+Point L:Point_struct # This is my Point structure # # *Example:* # # Creates a point $p$ of coordinates $(x=1,y=2)$. # # #+BEGIN_SRC julia :eval never :exports code # p=Point(1,2) # #+END_SRC # # You can add any valid Org-Mode directive. If you want to use # in-documentation link, use [[norm_link_example][]] # struct Point x::Float64 y::Float64 end #+Point # Creates Point at origin $(0,0)$ Point() = Point(0,0) #+Point,Method L:norm_link_example # A simple function that computes $\sqrt{x^2+y^2}$ # # *Example:* #!p=Point(1.0,2.0); #!norm(p) # # See: [[Point_struct][]] # norm(p::Point)::Float64 = sqrt(p.x*p.x+p.y*p.y) #+Method,Internal # An internal function # # For symbol that are not exported, do not forget the "Foo." prefix: #!p=Point(1.0,2.0) #!Foo.foo(2.0,p) foo(r::Float64,p::Point) = Point(r*p.x,r*p.y) end
The documentation template is very simple. Before each item you want to document add these comment lines:
#+Tag1,Tag2,... L:an_extra_link_if_required # # Here you can put any Org mode text, for instance $sin(x)$ # #!sin(5) # julia code to be executed # # [[internal_link][]] struct A_Documented_Struct ... end
- #+Tag1,Tag2,… is mandatory, “#+” is followed by a list of tags. Later when you want to extract doc you can do filtering according these tags.
- L:an_extra_link_if_required is not mandatory. It defines a reference if you want to create doc links. The previous statement defines a link target named an_extra_link_if_required.
- [[internal_link][]] creates a link to a previously defined L:internal_link.
- !sin(5) will execute Julia code and include the output in the doc.
Also note that you can keep compatibility with docstring as follows:
""" foo() foo function... """ #+Tag # # foo function foo() = ...
3 Org-Mode side
You need Org-Mode plus ob-julia.el to be installed. For illustration we use this minimal Org-Mode document:
#+PROPERTY: header-args:julia :session *my_session* :exports code :eval no-export #+OPTIONS: ^:{} #+TITLE: Getting Started with a minimal example #+BEGIN_SRC julia :results output none :eval no-export :exports none using J4Org initialize_boxing_module(usedModules=["Foo"]) documented_items=create_documented_item_array("Foo.jl") #+END_SRC * Example Prints all documented items, except those tagged with "Internal" #+BEGIN_SRC julia :results output drawer :eval no-export :exports results print_org_doc(documented_items,tag_to_ignore=["Internal"],header_level=0) #+END_SRC
- using J4Org uses this package
- initialize_boxing_module(usedModules=[“Foo”]) defines what are the modules to use when executing Julia code extracted from the doc (the “#!” statements). Here we are documenting the
Foomodule, hence we must use it. - create_documented_item_array(“Foo.jl”) creates the list of documented items from file “Foo.jl”. You can use a list of files or a directory.
- print_org_doc(documented_items,tag_to_ignore=[“Internal”],header_level=0) prints all documented items, except those tagged with “Internal”.
4 Result after html-export
When html-exported with Org-Mode this will generate this document:
struct Point
This is my Point structure
Example:
Creates a point
of coordinates
.
p=Point(1,2)You can add any valid Org mode directive. If you want to use in-documentation link, use norm(…)
Point()
Creates Point at origin
norm(p::Point)::Float64
A simple function that computes
Example:
p=Point(1.0,2.0); norm(p)2.23606797749979See: struct Point
5 Further information
You can visit J4Org.jl for further details. You can even install it (Pkg.add("J4Org")). I have used it to document DirectConvolution.jl (this Direct Convolution Package html page for instance).
Disclaimer: this package is still in early development.
Org Project Template
A short post to present a script that generates Org Mode projects templates. I am not sure that It can be used as it without extra configuration, however if you are interested you can give it a try.
To use it, simply clone the Project-Template-For-OrgMode GitHub repo. Before doing anything you can have a look at the generated template:
firefox Project-Template-For-OrgMode/docs/index.html
Now if you want to create your own project, simply type:
./create_orgMode_project ProjectName
This will create a directory ProjectName/ filled with the template. Again you can check the result with:
firefox ProjectName/index.html
Then use Emacs + OrgMode as usual.
Again, I m not sure if it can be useful for someone else, but it is the tool I routinely use to quickly start new projects/notes…
If you think it can be useful for you, do not hesitate to contact me in order to see what can be improved.
Julia with Emacs Org mode
CAVEAT (Tue Jan 15 09:19:56 CET 2019)
ob-julia is no longer supported (see emacs-org-mode-workflow for instance). One must now use Jupyter-mode with Julia kernel.
By consequence this post is now obsolete and does not work anymore. The right thing to do now is to use Jupyter-mode with Julia kernel. I will try to fix this post as soon as I will have the time to do it.
Introduction
This post details how to use Emacs Org mode to create Julia notebooks and to perform HTML or PDF exports. I tried to get the simplest working solution.
Julia notebook functionality works out of the box thanks to ob-julia.el and this is what I am using instead of Jupiter notebooks. However, the solution to export HTML and PDF is not straightforward.
I wanted to:
- have nice Julia code snippets with full UTF8 supports,
- being able to export in both HTML and PDF, including bibliography.
I get a solution which is certainly not perfect, ideas to improve it are welcomed.
There are two points to take care of:
- LaTeX does not fully support UTF8, nor its listings package.
- to make the bibliography exportable in both HTML and PDF without .org file modification, I had to use a little trick.
The proposed solution uses:
- ob-julia.el : to support notebook functionality,
- ox-bibtex.el : used for html-export of the bibliography, requires bibtex2html (Debian package),
- luatex : under Debian, included in the texlive-latex-base package,
- biber : under Debian, the biber package,
- pygments : under Debian, the python-pygments package. Attention with python3-pygments, it does not on my computer. I have not investigated this.
Maybe I have forgotten something, just tell me (I am only using Linux).
There is a GitHub repository to reproduce results of this post. The example.pdf generated file is also present.
Emacs configuration
Getting ob-julia.el and ox-bibtex.el
You can found ob-julia.el and ox-bibtex.el in Org-mode Contributed Packages. Easy download can be performed using:
curl -o emacs_files/ob-julia.el https://code.orgmode.org/bzg/org-mode/raw/master/contrib/lisp/ob-julia.el curl -o emacs_files/ox-bibtex.el https://code.orgmode.org/bzg/org-mode/raw/master/contrib/lisp/ox-bibtex.el
Minimal init.el file
This is a minimal configuration to reproduce the results. The code with its comments is self-explaining:
;; Use your own packages for classical stuff (package-initialize) ;; requires Emacs speaks statistics, Org (require 'ess-site) (require 'org) ;; removes ugly horizontal lines in html-exported code ;; (not mandatory) (setq org-html-keep-old-src t) ;; As ob-julia.el and ox-bibtex are less common, ;; we use a local repository. ;; ;; Usage: emacs -q --load emacs_files/init.el ;; ;; In a more usual setting one should use: ;; (require 'ob-julia.el) ;; (require 'ox-bibtex) (load-file "emacs_files/ob-julia.el") ; works with ess-site, our notebook engine (load-file "emacs_files/ox-bibtex.el"); used for bibliography HTML-export ;; allows julia src block (requires ob-julia.el) (setq org-confirm-babel-evaluate nil) (org-babel-do-load-languages 'org-babel-load-languages '((julia . t))) ;; defines image width in the OrgMode buffer (this is not for html ;; exports, for this you must use #+HTML_ATTR: :width 900px for ;; instance) ;; ;; This is not mandatory, but useful when one uses the gr() Plots.jl ;; backend as it exports wide .png files. CAVEAT: use imagemagick for ;; image resizing. ;; (setq org-image-actual-width (/ (display-pixel-width) 4)) ;; uses the minted package instead of the listings one (setq org-latex-listings 'minted) ;; defines how to generate the pdf file using lualatex + biber (setq org-latex-pdf-process '("lualatex -shell-escape -interaction nonstopmode -output-directory %o %f" "biber %b" "lualatex -shell-escape -interaction nonstopmode -output-directory %o %f" "lualatex -shell-escape -interaction nonstopmode -output-directory %o %f"))
.org file configuration
For demonstration purpose we define an .org file example. This file is kept very simple to do not distract from the required configuration part.
LaTeX directives
We have the LaTeX configuration part:
# uses minted package instead of listings
#+LATEX_HEADER: \usepackage{minted}
# uses fonts to support Greek letters etc...
#+LATEX_HEADER: \usepackage{fontspec}
#+LATEX_HEADER: \setmonofont{DejaVu Sans Mono}[Scale=MatchLowercase]
# defines the \begin{comment} \end{comment} environment, used to avoid
# conflict between bibtex and biblatex
#+LATEX_HEADER: \usepackage{verbatim}
# uses the biblatex package (and not the old bibtex)
#+LATEX_HEADER: \usepackage[backend=biber, bibencoding=utf8 ]{biblatex}
# our bibliography file
#+LATEX_HEADER: \addbibresource{my-bib.bib}
We then define our the Julia code highlight style. This style is used by minted for PDF export.
#+BEGIN_EXPORT latex
\definecolor{bg}{rgb}{0.95,0.95,0.95}
\setminted[julia]{
bgcolor=bg,
breaklines=true,
mathescape,
fontsize=\footnotesize}
#+END_EXPORT
Our notebook
Now this is the beginning of our notebook. One can use Org as usual…
#+TITLE: My title
#+AUTHOR: author
* Very simple demo
#+BEGIN_SRC julia :eval no-export :session *demo_session* :exports none
using Plots
#+END_SRC
** UTF8 support + escape math equation
Note that UTF8 is supported (the \alpha variable) :
#+BEGIN_SRC julia :eval no-export :session *demo_session* :exports both :results silent :wrap "SRC julia :eval never"
# Generate a matrix $a_{i,j}=\mathcal{U}([0,1[)$
α=rand(4,5)
#+END_SRC
** Long lines are wrapped
#+BEGIN_SRC julia :eval no-export :session *demo_session* :exports both :results output :wrap "SRC julia :eval never"
function ⊗(a::AbstractArray{T},b::AbstractArray{S}) where {T<:Number,S<:Number} kron(a,b) end;
β=rand(2,5);
γ = α ⊗ β
#+END_SRC
** Plot example
You can easily generate plots, one example from [[http://docs.juliaplots.org/latest/examples/pyplot/][Plots Julia package]],
is used to generate Figure [[PolarPlot]].
#+BEGIN_SRC julia :eval no-export :session *demo_session* :exports code :results silent
Θ = linspace(0,1.5π,100)
r = abs(0.1 * randn(100) + sin.(3Θ))
plot(Θ,r,proj=:polar,m=2)
#+END_SRC
#+BEGIN_SRC julia :eval no-export :session *demo_session* :results graphics :file example.png :exports results
savefig("example.png")
#+END_SRC
#+CAPTION: A polar plot.
#+ATTR_HTML: :width 900px
#+NAME: PolarPlot
#+RESULTS:
[[file:example.png]]
** Org with bibliography
\begin{align}
\label{eq:one_eq}
{\frac {d}{dt}}\iint _{\Sigma (t)}\mathbf {F} (\mathbf {r} ,t)\cdot d\mathbf {A} = & \iint _{\Sigma (t)}\left(\mathbf {F} _{t}(\mathbf {r},t)+\left[\nabla \cdot \mathbf {F} (\mathbf {r} ,t)\right]\mathbf {v}
\right)\cdot d\mathbf {A} - \\
& \oint _{\partial \Sigma (t)}\left[\mathbf{v} \times \mathbf {F} (\mathbf {r} ,t)\right]\cdot d\mathbf {s} \nonumber
\end{align}
Eq. \ref{eq:one_eq} is demonstrated in cite:Flanders1973.
Bibliography
Now we reach a little trick to support both HTML and PDF bibliography exports:
#+BEGIN_EXPORT latex
\printbibliography
#+END_EXPORT
#+BEGIN_EXPORT latex
\begin{comment}
#+END_EXPORT
#+BIBLIOGRAPHY: my-bib plain
#+BEGIN_EXPORT latex
\end{comment}
#+END_EXPORT
Explanation:
To export HTML bibliography, ox-bibtex does the job with only one directive:
#+BIBLIOGRAPHY: my-bib plain
However, for PDF export we do not want to use ox-bibtex, as it does not support UTF8. The solution is to wrap this directive into a comment section in the generated .tex code:
#+BEGIN_EXPORT latex
\begin{comment}
#+END_EXPORT
#+BIBLIOGRAPHY: my-bib plain
#+BEGIN_EXPORT latex
\end{comment}
#+END_EXPORT
Now we must tell LaTeX to use biblatex, this is done thanks to this directive:
#+BEGIN_EXPORT latex \printbibliography #+END_EXPORT
Putting everything together you get the proposed solution. This is certainly not the cleanest approach, but I have not found simpler.
The my-bib.bib file
For our example we need a small bibliography my-bib.bib file:
@article{Flanders1973,
doi = {10.2307/2319163},
url = {https://doi.org/10.2307/2319163},
year = {1973},
month = {jun},
publisher = {{JSTOR}},
volume = {80},
number = {6},
pages = {615},
author = {Harley Flanders},
title = {Differentiation Under the Integral Sign},
journal = {The American Mathematical Monthly}
}
Usage
You can visit the GitHub repo to reproduce the results.
Starting Emacs with the local configuration
From project root directory type
emacs -q --load emacs_files/init.el
to start a new Emacs with our local configuration.
Recomputing the notebook
As I potentially have several notebooks to publish I have used the :eval no-export argument. By consequence the notebooks are not evaluated each time you publish but only once. If you want to recompute everything every time, simply remove this option. You can also use the :cache option.
By consequence, before exporting you must begin by a first evaluation of the notebook. Visit the example.org buffer and do M-x org-babel-execute-buffer (or use the C-c C-v b shortcut). Attention, be sure that Plots.jl is installed.
ERROR: MethodError: no method matching start(::…)
In the ∗demo_session∗ Julia session buffer you will certainly see this error:
ERROR: MethodError: no method matching start(::...)
This is not our fault, but a known problem (that would need a fix) julia-print-commands-not-working-in-emacs-org-mode. It does not affect the computed result (but only the output processing). To get the right output (without the error message) one workaround is to restart computation of the source block (C-c C-c).
Exporting
Still from the example.org buffer, you can do:
- HTML export with: C-c C-e h o
- PDF export with: C-c C-e l o
This should generate and open fresh hmtl and pdf files.
Note: concerning html files, this is a basic export, you can use your own HTML theme.
PixorBlog, C++, Computations, Linux 