perl-String-Tokenizer - A simple string tokenizer

Property Value
Distribution ROSA Enterprise Desktop
Repository ROSA Contrib Updates x86_64
Package name perl-String-Tokenizer
Package version 0.50.0
Package release 1-rosa.lts2012.0
Package architecture noarch
Package type rpm
Installed size 24.31 KB
Download size 16.24 KB
Official Mirror
A simple string tokenizer which takes a string and splits it on whitespace.
It also optionally takes a string of characters to use as delimiters, and
returns them with the token set as well. This allows for splitting the
string in many different ways.
This is a very basic tokenizer, so more complex needs should be either
addressed with a custom written tokenizer or post-processing of the output
generated by this module. Basically, this will not fill everyones needs,
but it spans a gap between simple 'split / /, $string' and the other
options that involve much larger and complex modules.
Also note that this is not a lexical analyser. Many people confuse
tokenization with lexical analysis. A tokenizer mearly splits its input
into specific chunks, a lexical analyzer classifies those chunks. Sometimes
these two steps are combined, but not here.


Package Version Architecture Repository
perl-String-Tokenizer-0.50.0-1-rosa.lts2012.0.noarch.rpm 0.50.0 noarch ROSA Contrib Updates
perl-String-Tokenizer - - -


Name Value
perl(String::Tokenizer) == 0.50.0
perl(String::Tokenizer::Iterator) -
perl-String-Tokenizer == 0.50.0-1:2012.0


Type URL
Binary Package perl-String-Tokenizer-0.50.0-1-rosa.lts2012.0.noarch.rpm
Source Package perl-String-Tokenizer-0.50.0-1.src.rpm

Install Howto

  1. Enable ROSA Contrib Updates repository on Install and Remove Software
  2. Update packages list:
    # urpmi.update -a
  3. Install perl-String-Tokenizer rpm package:
    # urpmi perl-String-Tokenizer




2009-07-28 - J?r?me Quelin <> 0.50.0-1mdv2010.0
+ Revision: 401608
- rebuild using %%perl_convert_version
- fixed license field

See Also

Package Description
perl-String-Truncate-1.100.600-2-rosa.lts2012.0.noarch.rpm A module for when strings are too long to be displayed in
perl-String-Util-1.10.0-2-rosa.lts2012.0.noarch.rpm Handy string processing utilities
perl-Sub-Exporter-ForMethods-0.100.50-2-rosa.lts2012.0.noarch.rpm Helper routines for using Sub::Exporter to build methods
perl-Sub-Exporter-GlobExporter-0.2.0-2-rosa.lts2012.0.noarch.rpm Export shared globs with Sub::Exporter collectors
perl-Sub-Override-0.80.0-1-rosa.lts2012.0.noarch.rpm Perl extension for easily overriding subroutines
perl-Switch-2.160.0-3-rosa.lts2012.0.noarch.rpm A switch statement for Perl
perl-Syntax-Highlight-Engine-Kate-0.60.0-1-rosa.lts2012.0.x86_64.rpm A Plugin for Component-Pascal syntax highlighting
perl-Syntax-Highlight-Perl6-0.880.0-2-rosa.lts2012.0.noarch.rpm Perl 6 Syntax Highlighter
perl-Sys-HostIP-1.810.0-1-rosa.lts2012.0.noarch.rpm Perl module to get ip address related info
perl-Sys-Info-Base-0.780.0-1-rosa.lts2012.0.x86_64.rpm Base classes for Sys::Info
perl-Sys-Info-Driver-Linux-0.780.0-1-rosa.lts2012.0.x86_64.rpm Linux driver for Sys::Info
perl-Sys-Load-0.200.0-3-rosa.lts2012.0.x86_64.rpm A perl5 module that indicate system load
perl-Sys-Mknod-0.20.0-2-rosa.lts2012.0.noarch.rpm Sys::Mknod - make special files
perl-Sys-Proctitle-0.40.0-2-rosa.lts2012.0.x86_64.rpm Sys::Proctitle - modify proctitle on Linux
perl-Sys-SigAction-0.150.0-1-rosa.lts2012.0.noarch.rpm Perl extension for Consistent Signal Handling