mirror of
https://github.com/go-gitea/gitea.git
synced 2024-11-01 08:47:40 -04:00
ad2642a8aa
* Implementation for calculating language statistics Impement saving code language statistics to database Implement rendering langauge stats Add primary laguage to show in repository list Implement repository stats indexer queue Add indexer test Refactor to use queue module * Do not timeout for queues
8 lines
313 B
Go
8 lines
313 B
Go
// Package tokenizer implements file tokenization used by the enry content
|
|
// classifier. This package is an implementation detail of enry and should not
|
|
// be imported by other packages.
|
|
package tokenizer
|
|
|
|
// ByteLimit defines the maximum prefix of an input text that will be tokenized.
|
|
const ByteLimit = 100000
|