EN ET

Installation

Installation Using Docker (recommended)

Note

Currently all installation examples for system dependencies are only shown for Debian-based systems.

Install Docker

The easiest way to run TEXTA Toolkit is to use prebuilt Docker images from TEXTA registries. For that, one needs to install Docker and docker-compose:

sudo apt-get update
sudo apt-get install docker docker-compose -y

To run Elasticsearch in Docker, one needs to increase maximum map count parameter for virtual memory:

sudo sysctl -w vm.max_map_count=262144

Configure docker-compose

The ecosystem is built using docker-compose. This is all defined in a file called docker-compose.yaml. To successfully run TEXTA Toolkit one needs to define following services:

• Elasticsearch for storing the documents (one can also run it without Docker)

• Redis for managing message queues in TK (one can also run it without Docker)

• MySQL or Postgres for storing user and project data (can also be run without Docker)

• TEXTA Toolkit RESTful API & GUI: docker.texta.ee/texta/texta-rest:latest

For executing aforementioned services one needs to have preconfigured docker-compose.yaml file:

version: '3'

# NOTE: This is an example setup. This is NOT a production-ready solution!

services:

  texta-elasticsearch:
    container_name: texta-elasticsearch
    restart: on-failure
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.7.2
    ports:
      - 9200
    environment:
     - bootstrap.memory_lock=true
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - elastic-data:/usr/share/elasticsearch/data

  texta-redis:
    container_name: texta-redis
    image: redis
    restart: on-failure
    ports:
    - 6379

  texta-mariadb:
    container_name: texta-mariadb
    image: mariadb:10.3.28
    restart: on-failure
    ports:
      - 3306
    environment:
      MYSQL_ROOT_USER: docker
      MYSQL_ROOT_PASSWORD: docker
      MYSQL_PORT: 3306
      MYSQL_ROOT_HOST: '%'
      MYSQL_DATABASE: docker
      MYSQL_USER: docker
      MYSQL_PASSWORD: docker
    volumes:
      - mariadb-data:/var/lib/mysql

  texta-rest:
    container_name: texta-rest
    image: docker.texta.ee/texta/texta-rest:latest
    ports:
      - 80:80
    env_file:
      - ./env
    volumes:
      - toolkit-data:/var/texta-rest/data
    depends_on:
      - texta-mariadb
      - texta-elasticsearch
      - texta-redis

volumes:
  elastic-data:
  toolkit-data:
  mariadb-data:

The compose file is used in combination with env file located in the same directory as docker-compose.yaml containing environment variables needed for operating TEXTA Toolkit:

# Service URLs
TEXTA_ES_URL=http://texta-elasticsearch:9200
TEXTA_REDIS_URL=redis://texta-redis:6379/0
TEXTA_API_URL=http://localhost

# General
TEXTA_TEST_LIVE_SERVER_PORT=7000
TEXTA_DEBUG=False
TEXTA_LANGUAGE_CODES=et,en,ru

# TK database settings.
DJANGO_DATABASE_ENGINE=django.db.backends.mysql
DJANGO_DATABASE_NAME=docker
DJANGO_DATABASE_USER=docker
DJANGO_DATABASE_PASSWORD=docker
DJANGO_DATABASE_HOST=texta-mariadb
DJANGO_DATABASE_PORT=3306

# UAA settings for TK
USE_UAA=False
TEXTA_UAA_REDIRECT_URI=http://localhost/api/v2/uaa/callback
TEXTA_UAA_FRONT_REDIRECT_URL=http://localhost/#/oauth/uaa
TEXTA_UAA_URL=http://localhost:8080/uaa
TEXTA_UAA_CLIENT_ID=login
TEXTA_UAA_CLIENT_SECRET=loginsecret

TEXTA_UAA_SCOPE_PREFIX=texta
TEXTA_UAA_SCOPES=openid texta.*
TEXTA_UAA_PROJECT_ADMIN_SCOPE=texta.project_admin
TEXTA_UAA_SUPERUSER_SCOPE=texta.admin

# Elastic authentication
# Only usable with the X-Pack security
#TEXTA_ES_USER=elastic
#TEXTA_ES_PASSWORD=changeme

# Superuser password
TEXTA_ADMIN_PASSWORD=1234

Run TEXTA Toolkit

Note

Before running TEXTA Toolkit, one should have 2 files in current working directory:

• docker-compose.yaml

• env

With the docker-compose.yaml and env files present and accounted for, pull the images and start the services by executing the following commands in the same directory with docker-compose.yaml and env:

sudo docker-compose pull
sudo docker-compose up

For running in detached mode, use the -d flag:

sudo docker-compose up -d

Note

Default admin password is defined in the enviroment variable TEXTA_ADMIN_PASSWORD=1234. You can change this to a more suitable password. Without the environment variable the default password is generated automatically and can be obtained from the container log.

When running, TEXTA Toolkit’s GUI should be available at:

• http://localhost

and API at:

• http://localhost/api/v1/

For stopping the services:

sudo docker-compose down

Installation for Development

If you want to develop TEXTA Toolkit, want more control over how you run it, or seriously dislike Docker, it is also possible to install Toolkit using Anaconda (or Miniconda as shown in following examples).

Install Miniconda

First one needs to download and install Miniconda to manage Python environments. Miniconda is chosen over Anaconda because it’s smaller in size, but Toolkit works well with both.

wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
sh Miniconda3-latest-Linux-x86_64.sh

Also, let’s permanently put Conda binaries to PATH variable:

echo 'export PATH=/path/to/miniconda3/bin:$PATH' >> ~/.bashrc

Clone the Repository

Now, let’s clone the repository and move to texta-rest direcory:

git clone https://git.texta.ee/texta/texta-rest
cd texta-rest

Build the Environment

Now that environment.yaml file from the texta-rest repository is present, let’s use it to build the environment. After the environment is built, let’s activate it:

conda env create -f environment-nogpu.yaml
conda activate texta-rest

Note

If you want Toolkit to use GPU, use environment-gpu.yaml instead.

Migrate & Run Toolkit

After the environment has been created and activated, let’s prepare the database:

python migrate.py

Running the migrate.py script will prepare Django migrations and execute them. It is necessary to run the script every time the source code changes because Toolkit’s data model might have changed as well. It is also required to run the script before using Toolkit for the first time to create both superuser account and database used to store the data model.

Note

Default admin password is defined in the enviroment variable TEXTA_ADMIN_PASSWORD=1234. You can change this to a more suitable password. Without the environment variable the default password is generated automatically and can be obtained from the container log.

Users can overwrite the username by running python migrate.py -u {{username}} instead. In cases where the passwords needs to be overwritten, running python migrate.py -o will overwrite the password with whatever value is in the environment variable TEXTA_ADMIN_PASSWORD.

Finally let’s run the development server:

python manage.py runserver

And in another terminal (with same conda environment and in the same folder) run the Celery worker responsible for asynchronous tasks:

celery -A toolkit.taskman worker -l info

Browsable API

• http://localhost:8000/api/v1/

TODO

API Reference

Reference for Toolkit API is available when running the Toolkit:

• http://localhost:8000/api/v1/swagger/

• http://localhost:8000/api/v1/redoc/