Locic 1.2 Very soon

The next release of the Loci Compiler Tools will soon be available!

Full release notes will be part of the provided locic-1.2 archives (and up on http://loci-lang.org) but (to briefly mention it) the main feature of this release is predicates, which is essentially a much clearer and easier to use form of C++’s proposed concepts feature.

Predicates are just boolean expressions (i.e. that evaluate to either true or false) depending on template variables. For example:

Predicates come in three forms:

  • Const predicates – Specify that a variable or value is const depending on the outcome of a predicate. The common case is methods that return a non-const result if their parent is non-const, but return a const result if their parent is const.
  • Noexcept predicates – Specify that a function or method throws depending on the outcome of a predicate. For example, a method may only be noexcept if another method it calls is noexcept.
  • Require predicates – Specify that a type/function/method is only valid depending on the outcome of a predicate. For example, an array is only copyable if its elements are copyable.

This kind of functionality and capability is far ahead of C++; combined with the design of Loci templates (being usable across compilation boundaries!) it provides an extremely clear and concise way to write structured code. It’s an important core aspect of the Loci type system and substantial portions of the standard library rely on this (and that will be an increasing trend).

Named Predicates

For the next release I’m planning to augment this functionality to allow specifying named predicates. Right now you can create type aliases like:

My idea is to extend this to predicates:

The vast majority of the compiler machinery is already in place for this sort of feature and there is a clear gain in readability, so I expect this to be one of the first features I work on in the next release cycle (1.3).

Remaining Work

So the compiler is now mostly ready for release (release date is expected to be this weekend). The remaining work is:

  • Improvements/fixes to the standard library.
  • Updating documentation.
  • Adding more tests.
  • Clearing up some of the messier parts of the code.

Dropped Features

Unfortunately exception specifications are going to be dropped from this release and pushed forward. Fortunately on a relative basis they aren’t a particularly important feature and they also have a slight conflict with noexcept predicates that should be resolved. I think people (including myself) care a lot more about other features (e.g. lambdas, import-from-export generator tool, better standard library) so I’m going to push the implementation of exception specifications into the more distant future.

Next release (1.3)

I’m expecting the next release to be focused on even further development of templates in the form of value templates (already mostly ready in the compiler), variadic templates and template argument deduction.

By adding these, functions, methods, interface methods etc. will become primitive types (i.e. like ints, floats, lvals etc.) and hence this will remove a nasty (and probably slightly broken) special case from the compiler. Hence developers will able to treat functions in the same way as other types (e.g. putting them in containers). The other (slightly simpler) new primitive type will be a statically sized array type (like C++’s std::array).

I’ll also start building a ported version of OpenP2P and in the process start building (alpha quality) tools that make it easier to create C or C++ to Loci bindings (i.e. calling from Loci to C and C++). The idea for this is to use Clang to analyse C and C++ source code and produce both some C++ code (for C this isn’t necessary) and a Loci ‘header’ file with the relevant imports. In the C++ case we need to generate some C++ code that generates some ‘export C’ (i.e. unmangled, standard C calling convention) functions that are callable from Loci and call the relevant C++ methods.

The C case is fairly simple but for C++ there are some issues, of course:

  • C++ templates require compile-time expansion. They also don’t express any constraints that could be translated to require() predicates.
  • C++ classes must have a compile-time known size (Loci classes have fixed size, but in the general case this size doesn’t need to be known until run-time) – This is actually not a problem for calling from Loci into C++, but would raise issues for the reverse situation (solution is probably just using PIMPL idiom, meaning a heap allocation, which is what C++ programmers would do anyway).
  • C++ supports function/method overloading.

These problems can all be solved, probably via some sort of human direction (i.e. specify that particular template instantiations be created, indicate names for overloaded methods). Overall with the help of Clang it should ultimately be possible to do the vast majority of the work automatically and leave the developer to clear up the odd cases.

In terms of release dates, I’m aiming for mid-Summer (i.e. around July).

Moving projects to Loci

It’s always been my intention to develop the Loci programming language as a tool that I would regularly deploy myself and this year that’s going to start happening!

The language and compiler have reached an early stage of maturity, with a substantial mix of implemented language features and a small (but quickly growing) set of useful standard library modules. Hence this seems the right time to move some of my projects, currently written in C++, over to Loci, both to take advantage of the strengths of the language, to guide future design decisions of the language and as a real-world test of the compiler. Chain Reversi has already been ported to Loci and has been particularly useful when optimising the compiler; I consider quick build times extremely important and I’m pleased to say both that the design of Loci means it can be expected to compile much faster than similar C++ code and that this is borne out in practice.

The first project to move over is my ‘second’ (but slightly neglected) project, OpenP2P, which is basically a set of libraries that make it easy to develop applications that use peer-to-peer networking. One of the aspects that held OpenP2P back was the lack of good event notification/handling in C++ (which was basically implemented as a layer over boost::asio) but fortunately Loci has std.event, which in my opinion follows a very clean approach to event signalling. So I’ll be moving OpenP2P across and integrating it neatly with the standard library modules.

There’s another reason for making this change. Loci has presented me with a couple of problems:

  • It’s too much fun to work on!
  • It’s too successful!

Hence it’s starting to dominate my projects time, which has led to it becoming relatively well structured but on the other hand my other projects have been neglected. By moving these projects over to Loci I get to work on both at the same time! I also get the thrilling feeling of the last 5 years of designing Loci and building the compiler coming to fruition in a very practical way.

The Internet discovers Loci

So it seems that in the last couple of days postings have appeared about Loci on Y Combinator and Reddit (first and second).

It’s really awesome to see that developers are interested in Loci and various questions have been asked about the language, which will help me to improve the documentation to better explain how the language works and the reasons behind the design decisions. The 1.2 release of Loci will be appearing in the next couple of weeks; it looks likely that there’ll be a desperate rush next week to get some great features into the compiler, and this release also includes some optimisation of Semantic Analysis (whereas 1.1 involved an enormous performance improvement to Code Generation).

If you’re reading this and you have a question, a suggestion and/or constructive criticism of the language, I recommend raising an issue on the GitHub repository and I’ll do my best to respond promptly; at some point I’ll probably set up a better system for discussion/questions (I’m thinking about a forum on the Loci website).

Locic Website is now documentation

So after much consideration I’ve just gone ahead and replaced the Loci website with the compiler’s own generated HTML documentation. This is after the realisation that since Sphinx and reStructured Text are such amazingly convenient tools I’ve ended up writing a huge amount about the language and the compiler. The original website used WordPress and it was tedious to update both the website and the documentation. Also with a bit of theming the Locic documentation actually looks really good.

Hopefully this means anyone discovering the website will be able to get a clear idea of the language by just visiting the website; it’s obviously tedious to actually get the compiler source and to build the documentation from source.

Right now the documentation is mostly up-to-date. There are a few sections that are ahead of the implementation (!), so that should probably be fixed. For example, there’s a section about exception specifications but they don’t actually exist in the compiler. Fortunately the features affected tend to be fairly small and so unlikely to affect users. Realistically the main issues from a user’s point of view are platform support, standard library development (I’ve already done loads on this and actively adding more functionality) and easier integration with C and development tools.

Locic moves to GitHub + Travis

The Loci compiler is currently in the process of being moved to a GitHub repository and built by Travis CI (Continuous Integration). If you visit the second link you can see the latest build status of Locic. This is particularly useful because it verifies the compiler builds against all the supported LLVM versions (3.3, 3.4 and 3.5), and furthermore that Locic can be built by both Clang and GCC.

I’m keen to support multiple versions of LLVM because it makes it much easier for end users to build it on their system with whatever LLVM version is available. LLVM 3.6 is currently under development so it’s a moving target and hence they’re currently set as ‘Allowed failures’, but I hope to fix the builds at some point soon.

Easier local builds

As part of this I’ve also substantially improved the upstream dependency searching for LLVM and Clang. This means that it’s now possible to just install the normal llvm-dev and clang packages (e.g. via apt-get) and then when you build Locic the CMake script will just find those automatically. If you have your own local build of LLVM (you’d have to be quite keen…) then you can set the root directory of that build so it’s used instead.

Expanding platform support

All of this is likely to be paired with a greater focus on getting the compiler to build on Mac and Windows (currently only Linux), and adding support for more ABIs in llvm-abi. In fact I’d also like to push llvm-abi into its own repository so it’s usable for other LLVM-based compiler front-ends.

I expect I’ll explain this in more detail at some point and probably also add documentation for the project when it’s separated from Loci, because it’s a really interesting capability for other compiler developers to avoid re-implementing the ABI code manipulations required for LLVM. It’ll also make sense to mention it on the LLVM mailing lists so I can get some suggestions for improving it to handle some of the more exotic cases.

Locic 1.1 released!

So the second version of the Loci Compiler Tools is now available (see Loci Compiler), with the main new features being:

  • Switching from C++-like template expansion to use Template Generators (to allow templated APIs across module boundaries)
  • Module imports and exports
  • scope(success), scope(failure) and scope(exit)
  • noexcept
  • Type-templated functions/methods
  • Type aliases
  • assert and unreachable statements
  • Implicit and explicit casts between types using templated methods
  • Standard library memory allocators and smart pointers
  • Standard library containers
  • Standard library strings
  • Vastly improved performance, particularly for Code Generation.
  • A larger set of examples and updates to examples to demonstrate newly implemented features.
  • Significantly improved documentation in reStructuredText using Sphinx, which can generate multiple output formats including HTML and PDF.
  • A much larger set of integrated tests to check both accept and reject cases, as well as testing the standard library.

The release was delayed slightly from the mid-August estimate in order to add support for LLVM 3.5 (so that LLVM 3.3, 3.4 and 3.5 are all supported as backends for Locic), which was initially scheduled for release on the 25th August 2014.

LLVM 3.5’s release has since been re-scheduled for the start of September, so the Locic 1.1 release was modified and tested for compatibility with LLVM 3.5 RC3 (pulled from SVN), which is expected to be near-identical to the actual LLVM 3.5 release.

Locic 1.1 Soon

I’m currently working hard on the second release of ‘Locic’, the compiler tools for the Loci programming language. The initial target date was around mid-August and it looks likely that I will be able to release on schedule.

While 1.0 provides the key features of the language (classes, algebraic datatypes, templated types), this new release really fleshes out the language implementation (and the language itself!) to include lesser but nevertheless important features. The internal structure is also being improved thoroughly, to generate better error messages, better quality code, etc. Performance wise, the compiler is now around 50 times faster than 1.0 (tested on the Chain Reversi example), due mainly to optimisations focused on CodeGen. Testing is also much more thorough, with a much larger set of integration tests for accept/reject cases, and I’m planning to add some unit tests on top.

One of the most valuable improvements to the compiler in this release is the ‘template fix’. Specifically, you can now put a templated implementation in one module, a declaration in another module, and then just link* them together and everything will work nicely!

In other words, you can do this:

So the point here is that unlike C++ you can use a templated class or function without needing its implementation**. This is one of the many ground breaking features of Loci that enables:

  • No more avoiding Templated APIs; they can now be part of a stable API!
  • Switching between different implementations of a templated class/function by simply linking them together.
  • Much faster compile times.
  • Optimisations to more effectively balance trade-off of code size versus performance (i.e. it’s easier to generate multiple instantiations of a template than to merge them).
  • No need to release templated code to clients (in a commercial setting).

So now when users write templated constructs they don’t need to treat them any differently from normal constructs!

* You could generate object code and then use the platform linker. However, I’d strongly encourage the use of llvm-link followed by opt so that you can benefit from inter-procedural optimisation.

** Naturally some users will be interested in the implementation, and this will be explained fully in the language design document. To summarise, the compiler will generate vtables for each type and then effectively perform virtual calls as needed. If you then link two modules together and run opt on the result you’ll find that all the virtual calls are eliminated by inlining (unless the compiler determines it’s not beneficial, though currently it always inlines), generating similar (or better) code to what you’d write if you ‘manually’ instantiated the templates.

Locic Release

I recently made the first release of the Loci compiler, available to download here. This covers a very broad range of the language with extensive support for the key features, such as classes, templates, exceptions and algebraic datatypes; the website has more detail.

With this release I aim to provide a taste of the language, and as part of this it includes a full port of the Chain Reversi program I originally wrote in C++, by using SFML’s C binding (I’m now dropping the C++ implementation in favour of this). This example, along with the other examples and the suite of tests, demonstrates the simplicity and elegance of the language (as well as the short compile times).

There’s still a long way to go until the compiler and language are suitable for widespread production use, and the next release (v1.1, scheduled Summer 2014) will resolve a number of issues and complete the implementation of features introduced in this first release.

atoi() vs std::stringstream

I was recently asked to compare the functionality of atoi() and std::stringstream.

Here’s the code:

And here’s the output:

So it looks like the functionality of both is actually very similar (though note std::stringstream has many ways to modify its behaviour); unfortunately it seems the stream doesn’t set the result to zero when given an empty string.