Metadata-Version: 1.1
Name: bigcode-tokenizer
Version: 0.1.0
Summary: Tool to tokenize source code
Home-page: https://github.com/tuvistavie/bigcode-tools/tree/master/bigcode-tokenizer
Author: Daniel Perez
Author-email: tuvistavie@gmail.com
License: UNKNOWN
Download-URL: https://github.com/tuvistavie/bigcode-tools/archive/master.zip
Description-Content-Type: UNKNOWN
Description: # bigcode-tokenizer
        
        Tool to tokenize source code.
        
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
