Installation

This guide will walk you through installing WiMarka and its prerequisites on your system.

Prerequisites

Before installing WiMarka, ensure your system meets the following requirements:

Python Version

WiMarka requires Python 3.12 or higher. You can check your Python version with:

python --version

If you need to install or upgrade Python:

Windows/macOS: Download from python.org

Linux: Use your distribution’s package manager

# Ubuntu/Debian
sudo apt update
sudo apt install python3.12 python3.12-pip

# Fedora
sudo dnf install python3.12

Microsoft Visual Studio (Windows Only)

WiMarka depends on llama-cpp-python, which requires C++ build tools:

Download Visual Studio
During installation, select “Desktop development with C++”
Ensure CMake is included in the installation

Note

You can install Visual Studio Community Edition (free) or Visual Studio Build Tools.

The build tools include:

MSVC C++ compiler
Windows SDK
CMake

macOS/Linux: Build tools are typically pre-installed or available through package managers.

Installation Methods

Method 1: Install from GitHub (Recommended)

The easiest way to install WiMarka is directly from the GitHub repository:

pip install git+https://github.com/wimarka-uic/WiMarka.git

This command will:

Download the latest version from GitHub
Install all required dependencies
Set up the wimarka command-line tool

Method 2: Install from Source

For development or to access the latest features:

# Clone the repository
git clone https://github.com/wimarka-uic/WiMarka.git
cd WiMarka

# Install in development mode
pip install -e .

The -e flag installs WiMarka in “editable” mode, allowing you to modify the source code and see changes immediately.

Virtual Environment (Recommended)

It’s good practice to install WiMarka in a virtual environment:

# Create a virtual environment
python -m venv wimarka-env

# Activate the environment
# On Windows:
wimarka-env\\Scripts\\activate

# On macOS/Linux:
source wimarka-env/bin/activate

# Install WiMarka
pip install git+https://github.com/wimarka-uic/WiMarka.git

Verifying Installation

After installation, verify that WiMarka is correctly installed:

Test Python Import

from wimarka.main import wmk_eval
print("WiMarka imported successfully!")

Test CLI Command

wimarka --help

You should see the WiMarka help message with available options.

Dependencies

WiMarka automatically installs the following dependencies:

Core Dependencies

click >= 8.3.0 - Command-line interface framework
huggingface-hub >= 0.36.0 - Model downloading and management
llama-cpp-python >= 0.3.16 - LLM inference engine
transformers >= 4.57.1 - Transformer models

Data Processing

numpy >= 2.3.4 - Numerical computing
pandas >= 2.3.3 - Data manipulation
scikit-learn >= 1.7.2 - Machine learning utilities

Deep Learning

torch >= 2.9.0 - PyTorch deep learning framework

Utilities

joblib >= 1.5.2 - Serialization and caching
requests >= 2.32.5 - HTTP library

Troubleshooting

llama-cpp-python Installation Fails

If llama-cpp-python fails to install:

Windows:

# Ensure Visual Studio is installed with C++ tools
# Then try installing with CMake args:
CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python

macOS:

# Install Xcode Command Line Tools
xcode-select --install

# Then install WiMarka
pip install git+https://github.com/wimarka-uic/WiMarka.git

Linux:

# Install build essentials
sudo apt install build-essential cmake

# Then install WiMarka
pip install git+https://github.com/wimarka-uic/WiMarka.git

PyTorch Installation Issues

If PyTorch installation is slow or fails:

# Install PyTorch separately first
pip install torch --index-url https://download.pytorch.org/whl/cpu

# Then install WiMarka
pip install git+https://github.com/wimarka-uic/WiMarka.git

Permission Errors

If you encounter permission errors:

# Use --user flag
pip install --user git+https://github.com/wimarka-uic/WiMarka.git

# Or use a virtual environment (recommended)

Disk Space

WiMarka downloads language models which require significant disk space:

Minimum: 5 GB free space
Recommended: 10 GB free space

The models are cached in:

Windows: C:\\Users\\<username>\\.cache\\huggingface\\
macOS/Linux: ~/.cache/huggingface/

Uninstallation

To uninstall WiMarka:

pip uninstall wimarka

To also remove downloaded models:

# Remove model cache
# Windows
rmdir /s C:\\Users\\<username>\\.cache\\huggingface

# macOS/Linux
rm -rf ~/.cache/huggingface

Next Steps

Now that WiMarka is installed, proceed to the Quick Start Guide guide to run your first evaluation!