This post covers starting FeatureBase using a simple Docker compose file and ingesting a moderate amount of data using a Python script. A container is started running a standalone instance of FeatureBase and exposes port 10101 for querying and ingestion.
Ingestion is done using Python through the main.py file. You may also ingest data using the docker-consumer example in the featurebase-examples repo.
If you would like to start an instance of FeatureBase configured for running a cluster, see the docker-cluster example.
Check Out the Repo
Clone the FeatureBase examples repo in a terminal and change into the docker-simple directory:
git clone https://github.com/FeatureBaseDB/featurebase-examples.git
cd featurebase-examples/docker-simple
Create the Docker Network
Before starting the FeatureBase container(s), you will need to create a Docker network to be used by the services:
docker network create fbnet
Start the Services
Start the services using docker compose:
docker compose up
NOTE: If you have issues with docker compose, try disabling v2 by going into settings..general in Docker Desktop.
You should now have a container running:
Run the Insert Script
The script inserts "draws" of 81 different cards from Set the Game. The cards are represented with strings. For example, 3G#~ is shorthand for 3 green shaded squiggles.
Before you run the script, ensure you have the requirements installed:
pip3 install -r requirements.txt
Now run the script to insert data:
% python3 main.py
OUTPUT:
% python3 main.py
There are 1201000 existing records.
Enter the draw size (12,15,18,21,24...): 12
Enter the number of draws: 1000000
There are 1201000 total records..
There are 1301000 total records..
...a few seconds later...
There are 2101000 total records..
Generated a total of 1000000 draws.
Use the UI to Query with SQL
To check this worked, in your browser head over to http://0.0.0.0:10101 and run the following query:
select count(*) from simpledocker where setcontains(draw, '3G#~');
Try other queries:
select * from simpledocker where setcontains(draw, '3G#~') and setcontains(draw, '2G○~');
Alternate Ingestion Method
To insert data using a consumer on your local machine, you may either follow the docker-consumer example, or use the Community Guide to unzip FeatureBase locally.
The community download will include the molecula-consumer-csv binary, which you can use to insert data into this example's FeatureBase instance.
When you run FeatureBase in a container, you'll have to tell your local machine to map the featurebase hostname in the local host's /etc/hosts file. Make the following entry in hosts:
127.0.0.1 featurebase
Assuming you installed FeatureBase's binaries in the ~/featurebase/ directory, you can now run the IDK consumer directly:
