Elektra Initiative

Avi Alkalay <avi@unix.sh>

Linux, Open Standards Consultant

Turning Linux into a Trully Integrated Operating Environment

Future, Past, Present

Where the Desktop is going ?

Where the Underlying System is Going ?

Can we keep evolving the Desktop without evolving the Underlying OS structure ?

Current Configuration Panorama

KConfig Style

GConf Style

Do It Yourself

Java, UNIX

Commercial

No unified single namespace

No unified single API to access configurations

No integration, because configuration is Software's soul

Linux is today a 1 + 1 = 2 Operation System

Elektrified Panorama

KConfig Namespace

GConf Namespace

Java, UNIX

Commercial

System Namespace

Single system-wide namespace

Single infrastructure API

Integration through Software's soul

We are looking for a 1 x 1 = 1 status

Configuration System Utopia

Cross and Non-Cross Platform Software
Apache, Samba, KDE, /sbin/init, inittools

FileSystem

FileSystem

SingleFile

SingleFile

WRegistry

FileSystem

FileSystem

SingleFile

SingleFile

FileSystem

FileSystem

SingleFile

SingleFile

OpenDir

FileSystem

FileSystem

SingleFile

SingleFile

Backends

Elektra API & Namespace :: Abstraction Layer

Why Linux/UNIX Needs a Key DB

Difficult to package software that automatically integrates with other installed software, anywhere

Selfish configuration files format

Different location from distro to distro

Section "InputDevice"

Identifier "Mouse0"

Driver "mouse"

Option "Protocol" "IMPS/2"

Option "Device" "/dev/input/mice"

Option "ZAxisMapping" "4 5"

Option "Emulate3Buttons" "yes"

EndSection

In Fedora 2:

/etc/X11/xorg.conf

Other Distros:

God knows where...

How About This ?

Section "InputDevice"

Identifier "Mouse0"

Driver "mouse"

Option "Protocol" "IMPS/2"

Option "Device" "/dev/input/mice"

Option "ZAxisMapping" "4 5"

Option "Emulate3Buttons" "yes"

EndSection

Need human eyes and brains

Need a manual

. . . and vi

Each information bit is easily accessible

Understandable and editable by any program (without having to implement a config file compiler)

Ready for admin GUIs

Elektra's 7 Design Principles

Unix look'n'feel and standards to Namespaces

Available anywhere, anytime, even to early boot-stage programs like /sbin/init. Absolutely no dependencies on complex libraries, network services or language encodings.

C with POSIX, portable to Linux, BSD, Unix, Windows, Mac, etc.

Zero knowledge about key/value semantics.

Filesystem-like reliable security.

No daemon.

Open Storage (text based).

Key Hierarchy

Root for system wide keys

Current user's key tree

Avi's personal keys

Luciana's personal keys

Valeria's personal keys

The system/* tree is stored under /etc/kdb/

The user:$USER/* tree is stored under ~$USER/.kdb/

The user/* tree is a shortcut to current user's tree

Key Hierarchy :: system/...

Root for system wide keys

Equivalent to /etc/fstab

Equivalent to /etc/group

Keys with detected HW

Equivalent to /etc/inittab

Network info, interface IPs, etc

Root for application global keys

Application 1

Application 2

Equivalent to /etc/passwd

Key Hierarchy :: user/...

Root for user-wide keys

User's environment (instead of ~/.bashrc)

User's aliases

First env var set

Second env var set (that depends on the first)

Example of $PATH being set

Third environment set

Application keys for this user

Temporary keys

Key Names

This little dot makes inactive the entire “MSN/*” subtree

Key Properties

Key

Name

Value

Value type

User domain or owner

Description/Comment

System UID & GID

Access Permissions

Active or inactive key

Last change time

Last access time

system/users/root/shadowPassword = $1$yM93nU

user:valeria/env/env2/PATH = $PATH:/usr/sbin

Elektra Library Architecture

Library Generic Linkage

Application

/lib/libelektra.so

libelektra-{backend}.so

Pure dynamic (in)dependency

Invisible by ldd

Defined per-process by environment $KDB_BACKEND

Dependency not propagated to the application

Dynamic single dependency

Visible by ldd

libelektra.so hides all I/O details

Application process space

Example of Local Backends

Application

/lib/libelektra.so

libelektra-filesys.so

libelektra-berkeleydb.so

libelektra-fstab.so

filesys backend

Leverages filesystem security model (ACLs) and hierarchy

Security delegated to the Kernel

berkeleydb backend

Small storage footprint

Leverages Berkeley DB transactions

fstab backend

Interprets /etc/fstab file as a container of key/value pairs

Possible other backends

ini-like

xorg.conf

passwd/group

smb.conf

Application process space

Daemon/Network Architecture

Application

/lib/libelektra.so

libelektra-daemon.so

Server

Protocol over a communication pipe

Client-Server Architecture

Credentials and authentication handled between the Server and libelektra-daemon.so

The Server can use another instance of libelektra.so to store the keys

May be a single point of failure

Application's space

Server's

While GConf is Fat and Dependent...

bash$ ldd /usr/lib/libgconf-2.so.4

libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0

libORBit-2.so.0 => /usr/lib/libORBit-2.so.0

libm.so.6 => /lib/tls/libm.so.6

libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0

libdl.so.2 => /lib/libdl.so.2

libgthread-2.0.so.0 => /usr/lib/libgthread-2.0.so.0

libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0

libpthread.so.0 => /lib/tls/libpthread.so.0

libc.so.6 => /lib/tls/libc.so.6

libpopt.so.0 => /usr/lib/libpopt.so.0

/lib/ld-linux.so.2

Elektra is Light . . .

bash$ ldd /lib/libelektra.so

libc.so.6 => /lib/tls/libc.so.6

/lib/ld-linux.so.2 => /lib/ld-linux.so.2

Ready to be used in a restrictive environment (no net, no OS services), even by /sbin/init

May have no daemon, so

No single point of failure

One process can't harm another's affairs with the key database

No communication protocol, so no extra fat here

No security holes, since all security is handled directly by the OS

Clean and simple

Key Database Administration

The kdb Command: Perfect for Scripts

bash$ kdb get system/filesystems/boot/mpoint

bash# kdb set system/sw/xorg/Screen/Display/Modes 1280x1024

bash$ kdb export system/sw/myapp > myapp.conf.xml

bash# kdb import < myapp.conf.xml

bash$ kdb edit -R user/sw/myapp

bash$ kdb ls -Rv system/sw/myapp

bash$ kdb monitor system/init/id/runlevel

bash$ kdb rm user/env/alias/vnc

bash$ kdb ln system/sw/myapp/key1 user/sw/myapp/key1

Full manual pages available !

XML Import and Export

bash$ kdb export user/env/alias > file.xml

bash$ kdb import < file.xml

Using kdb to Edit in Batch

Use any editor to edit in XML

Keys you add will be added

Keys you change will be changed

Keys you delete will be deleted

(thanks to the ksCompare() API method)

kdbe: The Elektra GUI Edit Tool

More friendly than the kdb command

Hierarchical view

Supports UNDOs and REDOs

If everything else fails, you can still directly edit the key database with vi, because it is all plain text !

The API

Ready to Be Used Today !

Develop today

C/C++

Shell

XML

Python

Ruby

Java

To come . . .

Perl

PHP

Others . . .

(any contribution for a language binding ?)

Only 3 Classes

KeyDB

Key

KeySet

Classes

C API Methods

Class KeySet

ksInit()

ksClose()

ksInsert()

ksAppend()

ksInsertKeys()

ksAppendKeys()

ksToStream()

ksCompare()

ksNext()

ksRewind()

keyInit()

keyClose()

keyIsInitialized()

keyNeedsSync()

keyIsSystem()

keyIsUser()

keyGetNameSpace()

keyIsDir()

keyIsLink()

keyToStream()

keyDup()

keyGetType()

keySetType()

keyGetNameSize()

keyGetFullNameSize()

keyGetName()

keySetName()

keyGetFullName()

keyGetFullRootName()

keyGetFullRootNameSize()

keyGetBaseName()

keyGetBaseNameSize()

keyGetRootName()

keyGetRootNameSize()

keyCompare()

keyGetUID()

keySetUID()

keyGetGID()

keySetGID()

keyGetCommentSize()

keyGetComment()

keySetComment()

keyGetAccess()

keySetAccess()

keyGetOwner()

keySetOwner()

keyGetDataSize()

keyGetString()

keySetString()

keyGetBinary()

keySetBinary()

keyGetLink()

keySetLink()

keyGetMTime()

keyGetATime()

keyGetCTime()

keyGetParentName()

keySetFlag()

keyGetFlag()

Class Key

kdbOpen()

kdbClose()

kdbGetValue()

kdbGetKeyByParent()

kdbGetKeyByParentKey()

kdbGetValueByParent()

kdbSetValue()

kdbSetValueByParent()

kdbRemove()

kdbLink()

kdbGetKeyByParent()

kdbGetKeyByParentKey()

kdbGetValueByParent()

kdbGetComment()

kdbSetComment()

kdbStatKey()

kdbGetKey()

kdbSetKey()

kdbGetChildKeys()

kdbGetRootKeys()

kdbSetKeys()

kdbMonitorKey()

kdbMonitorKeys()

Class KeyDB

The API is Fully Documented !!

http://elektra.sf.net/#man

API doc in browsable Doxygen format

API man pages

Many code examples

Current Ecosystem

Regedit GUI, for Key database editing

/sbin/init patch, instead of /etc/inittab

X.org patch, instead of /etc/X11/xorg.conf

KConfig and GConf backend (WIP)

NSSwitch module

Users database and authentication from Elektra

GLibC patches

fstab, resolver, etc

All those language bindings

Demo

The Elektra Initiative needs YOU !

Talk about it

Package, port it to other platforms

Write patches that Elektrifies old software

Use it in your own software

It's free. It's for you.

Thank You

IBM

Support for a personal Open Source project to make Linux better

Travel expenses, so I can show it to you, here, now

Hours in airports and flights, so I have time to code :-)

Elektra Initiative community

Multiple storage backends

Maintaining Subversion repository, etc

Patches, usefull software, language bindings, etc

XML works, kickoff for the Doxygen documentation

Elektra Initiative

Avi Alkalay <avi@unix.sh>

Linux, Open Standards Consultant

Turning Linux into a Trully Integrated Operating Environment

Backup Charts . . .

Events

About (pt_BR)

GNU/Linux, BSD e outros Unix atuais são uma soma de componentes intependentes coletados num bazar. Cada um já tem um sistema singular de configuração que não se integra com nenhum outro.

A Iniciativa Elektra tenta integrar as partes de software através de uma infraestrutura comum de configuração, representada por uma árvore hierárquica de chaves e valores (key/value pairs). Tenta transformar um sistema "1+1=2" em um "1*1=1".

Similar ao GConf, só que mais seguro, mais rápido e para todo o sistema.

2nd option

Hoje cada componente do SO tem um arquivo de configuração diferente.

Elektra é uma iniciativa para unificar e simplificar o zoológico de formatos de configurações de software, propondo uma árvore de chaves e valores universal, padronizada e portável, e uma bilioteca multiplataforma para seu acesso.

Requirements for the Elektra Project

Task force to build an integration ecosystem

Universal key-value pair namespace and API to store base system and user configuration atoms

Cross Platform

No dependencies, Unicode ready, secure, lightweight, POSIX compliant (highly portable)

Available anywhere, anytime, even to early boot stage programs like /sbin/init

Manageable by simple system tools like vi, cp, grep, etc. Plain text storage

What Elektra is Not

NOT Webmin, Linuxconf or YaST-like software

NOT client-server software

NOT an OS service that can become unavailable

Does NOT access SQL databases

NOT an alternative to LDAP or NIS

KeyDB class

Retrieve and commit Keys and KeySets, recursively or not

Retrieve and commit individual Keys value, by absolute name or relative to parent

Monitor and notify changes in Keys and KeySets

Create and delete regular, folder or symbolic link Keys

Things you can do with the Classes

Key class

Get and Set key properties like name, root and base name, value, type, permissions, changed time, description, etc

Compare all properties with other keys

Test if changed, if is a user/ or system/ key, etc

Flag it and test if key has a flag

Export to XML

KeySet class

Linked list of Key objects

Insert and append entire KeySets or Keys

Work with its internal cursor

Compare entire KeySets

Export to XML

Classes continued...

KDE, QT and Elektra

OS (Linux/BSD/Windows)

Elektra API

QT (class QConfig)

KDE (class KConfig)

KDE App

QT Application (cross platform)

See the presentation in this View Mode to see comments.

If you are going to deliver this presentation in an event, you can use this comments as a guide to what to talk in each page. Change the order, add pages, etc, to fit your way of thinking.

The real presentation mode has animations and it takes about 1 hour for me.

Lets face it: Linux/BSD/Unix today are completelly unintegrated.

Why ?

Because the soul or personality of a software are in its configuration files. Softwares, as people, should have affairs, integration, through their personality.

Open Source nature is to produce islands of projects: each one follows its own rules. So we have many config files format, APIs, and no unified namespace.

So a lot of ad-hoc integration must be done in the packaging time. This leads to different Linux distros to have different feels.

Homogeneous software configuration is key.

Elektra provides an infrastructure ready to be used by any type of software, and the key elements of its design are:

Single unified namespace for configuration atoms

Single simplest C API (and many other language bindings)

API and Namespace to be available even in the most restrictive environment, like when /sbin/init

Then, softwares can use it directly, or frameworks (Gnome, KDE) can rely on it as a backends.

But this is not enough.

Cross platform software, like Apache, need it everywhere. Its source code and name of configuration atoms it accesses, must be the same across platforms.

The same for cross-platform Java apps.

So one more requirement for Elektra is to be cross platform.

This leads to the requirement of a complete lack of dependencies, and strict POSIX compliency.

So lets have a deeper look on a real life problem.

Who knows what is this file? Everybody knows, because everybody had to edit it by hand once. And everybody was already asked by a newbie to help him with this file.

XF86Config, as 100% of Linux configuration files, were designed to be human readable and human editable.

A software can't read and automatically fully understand a text configuration file. This is what makes us, superior God-assembled creatures, different from machines.

So this is why YOU have to edit this silly file everytime you buy a new commodity video card, mouse, etc. And thats the same for modules.conf, httpd.conf, etc.

So instead of all those files that require human brains and vi, what if we create a standarized tree of key/value pairs, where every configuration atom is programatically accessible, and modifiable (security restrictions apply) ?

This way, any software can see how other software is configured (configuration is software's personality) and take more wise decisions for itself.

It is designed to Unix systems, by Unix guys, to Unix guys.

It can't be a solution only for some programs. It must be available anywhere, and has absolutely no dependencies.

Available to all systems, POSIX complient, Unicode, so then it can be used in all platforms by cross-platform software.

Elektra is stupid software by design. Otherwise it would be an endless project.

Security is handled like this: “Security is not my business. Is Kernel's”

Deamons are single point of failures and performance bottlenecks.

Unix sysadmins need vi, grep, etc for living.

The key tree is as simple as possible, and we have only 2 trees:

system/

user/

The user/ tree is dynamic and it is an alias to the current user tree, which full name is user:$USER/....

Having this user:$USER name, one user can access other users configurations (if he'll have permissions for that).

The system tree is used for system wide configurations, pretty much as /etc directory.

You are desincouraged to create keys or folders right under system/. This is reserved for very essential configurations.

The system/sw subtree is there to store higher level software keys.

The user tree is similar to the system tree, but is used for user specific configurations.

This is why its location on the system is under the user's home directory. If you delete a user, all its keys go away.

Some examples of key names and values. Their names are pretty much as filenames.

Activating the animation, we can see that a key name that begins with a dot is inactive or commented.

Other similarities with files are symbolic links, permissions, etc.

A Key has this properties, which are very similar to files on the filesystem.

About Elektra's complete lack of dependencies, lets compare it with the similar GConf, and show you why GConf can never be used for system wide and early boot stage programs. The same applyies to UniConf.

It is completelly dependent on Gnome libraries

Can't be used by early boot stage programs

It is a daemon, which leads to single point of failure (same problem of the Windows Registry)

Was not designed for cross user configurations (one user can't access other's keys)

Was not designed with security in mind

Is too somplex for super basic system programs

On the other hand, Elektra is agressively independent of everything. Not because it is statically linked, but because it is plain simple.

Being only a library, it does not handle security and access permissions. This is good, because this job 100% delegated to the underlying OS.

You can read above the other advantages.

So lets have an overview on the tools we have available today for the key database administration.

These are the things you can do with the kdb command. Some really cool ones are:

Create a key that is a symbolic link to another key

Monitor changes on keys

Edit many keys in one shot, with any editor

XML import and export

kdb is perfect for Shell scripts.

One supercool kdb feature is to let you import and export to a standarized XML.

This is usefull for

Transfering entire program's keys from one machine to another

Make backups

Package default configurations that will be imported at package installation time (as a post script)

Edit them with XML editors

kdb XML support was implemented to handle Unicode and different charsets. Actually, Elektra was implemented with this issue in mind.

Another supercool feature: Edit many keys in batch, represented as an XML document.

As you can see, we can use the editor we want, from simples vi, to even a true XML editor.

So if you're not a command line guy, you can leave kdb, and use Regedit, by Gregor Burger, to graphically edit Elektra keys.

(Activate animation)

And since we are talking about plain text files, we can still manage the key database with those good old Unix tools.

Highlights about the Elektra API...

Thanks to a strong ecosystem, you can already develop in these other languages.

If you are fluent in some other languge, please write and contribute Elektra bindings to it.

Althought it is pure C, it follows an Object Oriented design.

Elektra API is very simple, providing only 3 classes.

KeyDB, to access the key database

Key, as the entity for all key properties

KeySet, as a linked list of keys, for grouping and better handling

Speak out about more things you can do with the API based on the 2 pages located after the end of presentation.

These are most of the API calls provided.

While programming, you'll get help from a complete Doxygen documentation, full of code examples.

There are full API man pages (the man command), if you don't like to browse.

Elektra's ecosystem is growing everyday.

With only 6 months of projects, we have about 30 contributors, patchers, language binders, etc.

What makes a software strong is not its features, but a good ecosystem.

Elektra has both.

Hide the presentation and show some things:

The kdb command

Editing with “kdb edit” with different editors

Regedit

“kdb monitor”

NSS plugin with the users and groups database

Call for action !!

We need help !!!!

We need patchers !!

IBM is kindly paying travel expenses for me when I'm invited to talk in events.

These are the requirements that the Elektra Project fullfils.

Its development is oriented strictly by these points.

It can't be too complex and full of featured (schemas, too many value types, plugins, etc), because then it will have dependencies, and will be optimized only for high level software. And to optimize it for high level software means to deoptimize it for base system software. And this is not what we want.

When needed, you can allways stack specific functionality on top of it.

Please do not missunderstand Elektra with other types of software.

Before deveolpement begun, a lot of research was made on existing software to avoid reinventing the wheel.

And we found none that addressed our key aspects (say it again !):

Single unified namespace for configuration atoms

Single simplest C API

Absolutelly no dependencies

Availability everywhere everytime

These other types of softwares may complete Elektra's functionality.

Elektra Project

Linux as a wholly integrated system with Elektra

configuration integration registry hierarchy

http://elektra.sf.net

Avi Alkalay

2003-06-20T16:23:01

Avi Alkalay