2 Commits

Author	SHA1	Message	Date
Adil Zouitine	6a75b4761a	refactor(TokenizerProcessor): improve dependency handling and observation management ... - Updated TokenizerProcessor to conditionally import AutoTokenizer based on the availability of the transformers library, enhancing flexibility. - Modified tokenizer attribute type to Any to accommodate scenarios where transformers may not be installed. - Improved observation handling by using a more concise approach to manage the transition dictionary, ensuring compatibility with existing data structures. - Added error handling for missing transformers library, providing clear guidance for users on installation requirements.	2025-08-07 17:07:20 +02:00
Simon Alibert	d4ee470b00	Package folder structure (#1417) ... * Move files * Replace imports & paths * Update relative paths * Update doc symlinks * Update instructions paths * Fix imports * Update grpc files * Update more instructions * Downgrade grpc-tools * Update manifest * Update more paths * Update config paths * Update CI paths * Update bandit exclusions * Remove walkthrough section	2025-07-01 16:34:46 +02:00