python-tokenizers on AUR (Arch User Repository)

Last updated: December 27,2024

1. Install "python-tokenizers" effortlessly via AUR with the help of a helper(YAY)

a. Install YAY (https://github.com/Jguer/yay)


    
        
            $
        

        
            
                sudo pacman -S --needed git base-devel && git clone https://aur.archlinux.org/yay.git && cd yay && makepkg -si
            
                    
    
    Copied

b. Install python-tokenizers on Arch using YAY


    
        
            $
        

        
            
                yay -S
            
                        
                python-tokenizers
            
                    
    
    Copied

* (Optional) Uninstall python-tokenizers on Arch using YAY


    
        
            $
        

        
            
                yay -Rns
            
                        
                python-tokenizers
            
                    
    
    Copied

2. Manually Install "python-tokenizers" via AUR

a. Ensure you have the base development tools and git installed with:


    
        
            $
        

        
            
                sudo pacman -S --needed
            
                        
                base-devel git
            
                    
    
    Copied

b. Clone python-tokenizers's git locally


    
        
            $
        

        
            
                git clone https://aur.archlinux.org/python-tokenizers.git 
            
                        
                ~/python-tokenizers
            
                    
    
    Copied

c. Go to ~/python-tokenizers folder and install it


   
    
        
            $
        

        
            
                cd
            
                        
                ~/python-tokenizers
            
                    
    
    Copied

    

   
    
        
            $
        

        
            
                makepkg -si
            
                    
    
    Copied

3. Information about the python-tokenizers package on Arch User Repository (AUR)

ID: 1410688
Name: python-tokenizers
PackageBaseID: 172312
PackageBase: python-tokenizers
Version: 0.15.2-1
Description: Fast State-of-the-Art Tokenizers optimized for Research and Production
URL: https://github.com/huggingface/tokenizers
NumVotes: 3
Popularity: 0.28205
OutOfDate:
Maintainer: xiota
Submitter: filipg
FirstSubmitted: 1634987840
LastModified: 1708131535
URLPath: /cgit/aur.git/snapshot/python-tokenizers.tar.gz