DBToaster
License

The DBToaster technology is protected by pending US and international patents.

The DBToaster software is provided free of charge for evaluation, teaching, learning, and research purposes. If you would like to use DBToaster in any other form, contact us at

dbtoaster at epfl dot ch

Redistribution with or without modification is prohibited unless explicit permission is granted by the copyright holders.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

We plan to distribute future releases of DBToaster under a more relaxed license.

1. Benchmarking and Experimentation

You are invited to perform and report on benchmark results and experimental comparisons of your system against DBToaster, but we ask you for fairness and to run DBToaster with settings that allow it to perform at its best. Specifically, the DBToaster interpreter is provided for your convenience and is not to be used for timing purposes. For the purpose of comparison with other data management systems, please use the C++ rather than the Scala backend. Please contact us if you have questions about how to optimize the performance of DBToaster.

2. DBToaster Changelog

Release 2.1 (revision 3346) - 2014/11/06

  • Added an optimization for eliminating some temporary maps that were formerly used in IVM triggers.
  • Fixed a bug in generating IVM triggers for queries contaning OR operator in the WHERE clause.

Release 2 (revision 3263) - 2014/08/01

  • Implemented a new back-end from scratch in Scala for compiling M3 programs generated for SQL queries by DBToaster's front-end.
  • Used state-of-the-art compilation techniques for generating the most efficient view maintenance programs (in both Scala anf C++), in the new back-end.
  • Extensively used LMS (https://github.com/epfldata/lms/tree/booster-develop-0.3) as a light weight modular generative compiler for easily implementing domain-specific optimizations used in the new back-end.
  • Removed the deprecated flags in the new back-end.
  • Extended the definition of a ring base, new ring methods, modified rings of arithmetic and calculus values.
  • Removed the dependency on Boost for compiling the generated C++ programs. Now, any C++11 compliant compiler (either clang++ or g++-4.8 or above) can be used without needing other dependencies.
  • Implementing specialized base-types in C++ (PString and KDouble instead of standard string and double, for comparison reasons or a faster execution.)
  • Implemented more efficient collection types in both C++ and Scala.
  • Provided the correct scope when descending into AggSum, Lifts, Externals, and Exists.
  • Implemented a super efficient memory caching strategy in C++ collections to further improve the performance.
  • Changed unifiation to respect the scope of variables.
  • Applied more optimizations:
    • Added OPTIMIZE-PATTERNS option (active for O3) which avoids generating useless secondary indices.
  • Fixed the known bugs from the previous release:
    • Minor fixes in the SQL parser.
    • Fixed issues in generating maps with input variables, when it is not expected.
    • Fixed a duplicate group-by vars in AggSum.
    • Fixed the wrong order of aggregate targets.
    • Fixed an issue where optimizer was removing equality checks.
    • Fixed an issue related to generating unit values (Error: "Cannot produce one of type '').
    • Fixed mk_domain_restricted_lift.
    • Fixed a wrong materialization decision (appeared on TPCH-18).
    • Fixed an issue where external functions were formatted differently when appearing in DECLARE QUERY statements.

Release 1 (revision 2827) - 2013/02/11

  • Added support for more SQL features:
    • SELECT DISTINCT
  • Added support for standard functions in SQL queries.
  • Added support for different levels of view maintanance Depth-0, Depth1, ..., DepthN.
  • Added support for calculatiion of the final result upon user request ( EXPRESSIVE-TLQS).
  • Added more examples into the release.
  • Added bin/dbtoaster as wrapper script that handles the compilation process for C++ and Scala.
  • Added support for DELETE operations, as well as INSERTs.
  • Several changes to improve compatibility with Windows and speed up C++ and Scala compilation.
  • Code formatting of Scala and C++ source files.
  • Improved error reporting for SQL errors. Errors will include contextual information.
  • Minor changes to get the compilation of dbtoaster working on the latest CYGWIN.
  • Added new query interface for both Scala and C++.
  • Used actors for input processing in Scala.
  • Added -g flag to pass through an argument to the second-stage compiler.
  • Added DELETE-ON-ZERO flag for the Scala backend.
  • Added HEURISTICS-MINIMAL-MAPS debug flag to choose between materializing the whole relation or only the part that is needed.
  • Added flags CALC-NO-OPTIMIZE and CALC-NO-DECOMPOSITION to disable the calculus optimizations and graph decomposition, respectively.
  • Added HEURISTICS-ENABLE-INPUTVARS flag, which enables experimental support for incremental view caches.
  • Minor change in Heuristics to correctly pass the scope to nested subexpressions.
  • Applied more optimizations:
    • Removed unnecessary ifthenelse blocks where "if(pred) t else t" with. a side effect free predicate.
    • Introduced LookupOrElse operator for saving the unnecessary computations, when a tuple does not exist.
    • Changed the comparison of two AggSums expressions. This comparison now results in true if there is a mapping between the subexpressions and GB1 maps to GB2. This fix prevents generation of duplicate maps (e.g. AggSum([A,B], m[][A,B]) equals AggSum([D,C],m[][C,D]).
    • Changed the approach for dealing with nested filters.
    • Fixed a problem where a multiplication with a comparison would not result in using a filter operation, if the comparison was on the left-hand side.
    • More elegant handling of filter expressions on non-collections.
    • Optimization for instances of Aggregate(Filter(m)) where m is an inexpensive expression
    • Transformed (B^=dA)*R(dA,B) into R(dA,B)*(B=dA), in order to prevent expressions of the form R(dA,dA).
    • Added "HEURISTICS-PULL-OUT-VALUES" flag to prevent ValueRing terms and comparisons from being pulled inside materialized maps. In certain cases, this option might decrease the number of generated maps (e.g. mddb/query2.sql).
    • Expressions of the form "E * 1.0" and "E * 0.0" are being reduced to "E" and "0", respectively (note the float values), yielding some dead code.
    • Used long type for representing dates in generated programs.
    • Added Heuristics checks if the optimizer re-introduced AggSums in the expression being materialized.
    • Fixed the bug that was materializing the whole relation at depth less than infinity, and not only the needed columns (even when minimal_maps = true).
    • Added membership tests around lookups on LHS and RHS maps with input and output variables.
  • Fixed the known bugs from the previous release:
    • Clarified the unbound variable error.
    • Resolved problems with DB parser.
    • Extended the SQL parser to support semicolon in strings.
    • Fixed inconsistencies in the examples paths presented in the documentation.
    • Fixed the problem with creating temporary files.
    • Removed "long long" types from C++ generated code.
    • Fixed transactional inconsistency bug.
    • Added a few variable initializations to prevent reading uninitialized data.
    • Fixed warning about floating numbers not having a trailing 0.
    • Fixed the name collision problem of raw relations.

Beta for Release 1 (revision 2525) - 2012/07/19

The initial release of DBToaster.