tokenizer-git on AUR (Arch User Repository)

Last updated: March 11,2025

1. Install "tokenizer-git" effortlessly via AUR with the help of a helper(YAY)

a. Install YAY (https://github.com/Jguer/yay)


    
        
            $
        

        
            
                sudo pacman -S --needed git base-devel && git clone https://aur.archlinux.org/yay.git && cd yay && makepkg -si
            
                    
    
    Copied

b. Install tokenizer-git on Arch using YAY


    
        
            $
        

        
            
                yay -S
            
                        
                tokenizer-git
            
                    
    
    Copied

* (Optional) Uninstall tokenizer-git on Arch using YAY


    
        
            $
        

        
            
                yay -Rns
            
                        
                tokenizer-git
            
                    
    
    Copied

2. Manually Install "tokenizer-git" via AUR

a. Ensure you have the base development tools and git installed with:


    
        
            $
        

        
            
                sudo pacman -S --needed
            
                        
                base-devel git
            
                    
    
    Copied

b. Clone tokenizer-git's git locally


    
        
            $
        

        
            
                git clone https://aur.archlinux.org/tokenizer-git.git 
            
                        
                ~/tokenizer-git
            
                    
    
    Copied

c. Go to ~/tokenizer-git folder and install it


   
    
        
            $
        

        
            
                cd
            
                        
                ~/tokenizer-git
            
                    
    
    Copied

    

   
    
        
            $
        

        
            
                makepkg -si
            
                    
    
    Copied

3. Information about the tokenizer-git package on Arch User Repository (AUR)

ID: 1244682
Name: tokenizer-git
PackageBaseID: 131770
PackageBase: tokenizer-git
Version: r113.8b0c4e2-1
Description: Convert source code into numerical tokens
URL: https://github.com/dspinellis/tokenizer
NumVotes: 0
Popularity: 0
OutOfDate:
Maintainer: aksr
Submitter: aksr
FirstSubmitted: 1524406264
LastModified: 1682147002
URLPath: /cgit/aur.git/snapshot/tokenizer-git.tar.gz