python-tokenizers on AUR (Arch User Repository)

Last updated: November 24,2024

1. Install "python-tokenizers" effortlessly via AUR with the help of a helper(YAY)

a. Install YAY (https://github.com/Jguer/yay)

$ sudo pacman -S --needed git base-devel && git clone https://aur.archlinux.org/yay.git && cd yay && makepkg -si

b. Install python-tokenizers on Arch using YAY

$ yay -S python-tokenizers

* (Optional) Uninstall python-tokenizers on Arch using YAY

$ yay -Rns python-tokenizers

2. Manually Install "python-tokenizers" via AUR

a. Ensure you have the base development tools and git installed with:

$ sudo pacman -S --needed base-devel git

b. Clone python-tokenizers's git locally

$ git clone https://aur.archlinux.org/python-tokenizers.git ~/python-tokenizers

c. Go to ~/python-tokenizers folder and install it

$ cd ~/python-tokenizers $ makepkg -si

3. Information about the python-tokenizers package on Arch User Repository (AUR)

ID: 1410688
Name: python-tokenizers
PackageBaseID: 172312
PackageBase: python-tokenizers
Version: 0.15.2-1
Description: Fast State-of-the-Art Tokenizers optimized for Research and Production
URL: https://github.com/huggingface/tokenizers
NumVotes: 3
Popularity: 0.28205
OutOfDate:
Maintainer: xiota
Submitter: filipg
FirstSubmitted: 1634987840
LastModified: 1708131535
URLPath: /cgit/aur.git/snapshot/python-tokenizers.tar.gz