utils Package

bittorrent Module

Torrenting utils, mostly for handling bencoding and torrent files.

class flexget.utils.bittorrent.Torrent(content)

Bases: object

Represents a torrent

KEY_TYPE

alias of str

add_multitracker(tracker)

Appends multi-tracker to this torrent

encode()
classmethod from_file(filename)

Create torrent from file on disk.

get_comment()
get_filelist()

Return array containing fileinfo dictionaries (name, length, path)

get_info_hash()

Return Torrent info hash

get_multitrackers()

Return array containing all multi-trackers in this torrent. Returns empty array if torrent has only standard single announce url.

get_size()

Return total size of the torrent

private
remove_multitracker(tracker)

Removes passed multi-tracker from this torrent

set_comment(comment)
flexget.utils.bittorrent.bdecode(text)
flexget.utils.bittorrent.bencode(data)
flexget.utils.bittorrent.clean_meta(meta, including_info=False, logger=None)

Clean meta dict. Optionally log changes using the given logger.

See also http://packages.python.org/pyrocore/apidocs/pyrocore.util.metafile-pysrc.html#clean_meta

@param logger: If given, a callable accepting a string message. @return: Set of keys removed from C{meta}.

flexget.utils.bittorrent.decode_item(next, token)
flexget.utils.bittorrent.encode_dictionary(data)
flexget.utils.bittorrent.encode_integer(data)
flexget.utils.bittorrent.encode_list(data)
flexget.utils.bittorrent.encode_string(data)
flexget.utils.bittorrent.encode_unicode(data)
flexget.utils.bittorrent.is_torrent_file(metafilepath)

Check whether a file looks like a metafile by peeking into its content.

Note that this doesn’t ensure that the file is a complete and valid torrent, it just allows fast filtering of candidate files.

@param metafilepath: Path to the file to check, must have read permissions for it. @return: True if there is a high probability this is a metafile.

flexget.utils.bittorrent.tokenize(text, match=<built-in method match of _sre.SRE_Pattern object at 0x2e21d20>)

cached_input Module

class flexget.utils.cached_input.InputCache(**kwargs)

Bases: sqlalchemy.ext.declarative.Base

added
entries
hash
id
name
class flexget.utils.cached_input.InputCacheEntry(**kwargs)

Bases: sqlalchemy.ext.declarative.Base

cache_id
entry
id
class flexget.utils.cached_input.cached(name, persist=None)

Bases: object

Implements transparent caching decorator @cached for inputs.

Decorator has two parameters:

  • name in which the configuration is present in feeds configuration.
  • key in which the configuration has the cached resource identifier (ie. url). If the key is not given or present in the configuration :name: is expected to be a cache name (ie. url)

Note

Configuration assumptions may make this unusable in some (future) inputs

cache = {}
flexget.utils.cached_input.clear_cache(manager)

Clears the input cache before execution. This is neccessary for webui or otherwise it will only use cache.

flexget.utils.cached_input.config_hash(config)
Parameters:config (dict) – Configuration
Returns:MD5 hash for config
flexget.utils.cached_input.db_cleanup(session)

Removes old input caches from plugins that are no longer configured.

database Module

class flexget.utils.database.CaseInsensitiveWord(word)

Bases: sqlalchemy.ext.hybrid.Comparator

Hybrid value representing a string that compares case insensitively.

lower()
operate(op, other)
class flexget.utils.database.QualityComparator(expression)

Bases: sqlalchemy.ext.hybrid.Comparator

Database quality comparator fields which can operate against quality objects or their string equivalents.

operate(op, other)
flexget.utils.database.ignore_case_property(text_attr)
flexget.utils.database.pipe_list_synonym(name)

Converts pipe separated text into a list

flexget.utils.database.quality_property(text_attr)
flexget.utils.database.safe_pickle_synonym(name)

Used to store Entry instances into a PickleType column in the database.

In order to ensure everything can be loaded after code changes, makes sure no custom python classes are pickled.

flexget.utils.database.text_date_synonym(name)

Converts Y-M-D date strings into datetime objects

flexget.utils.database.with_session(func)

” A decorator which creates a new session if one was not passed via keyword argument to the function.

Automatically commits and closes the session if one was created, caller is responsible for commit if passed in.

flexget.utils.database.year_property(date_attr)

imdb Module

class flexget.utils.imdb.ImdbParser

Bases: object

Quick-hack to parse relevant imdb details

parse(imdb_id)
class flexget.utils.imdb.ImdbSearch

Bases: object

best_match(name, year=None)

Return single movie that best matches name criteria or None

ireplace(str, old, new, count=0)

Case insensitive string replace

search(name)

Return array of movie details (dict)

smart_match(raw_name)

Accepts messy name, cleans it and uses information available to make smartest and best match

flexget.utils.imdb.extract_id(url)

Return IMDb ID of the given URL. Return None if not valid or if URL is not a string.

flexget.utils.imdb.make_url(imdb_id)

Return IMDb URL of the given ID

log Module

Logging utilities

class flexget.utils.log.LogMessage(md5sum)

Bases: sqlalchemy.ext.declarative.Base

Declarative

added
id
md5sum
flexget.utils.log.log_once(message, logger=<flexget.logger.FlexGetLogger object at 0x378c110>)

Log message only once using given logger. Returns False if suppressed logging. When suppressed verbose level is still logged.

flexget.utils.log.purge(session)

Purge old messages from database

qualities Module

Warning

Many methods in this module are not thread safe since they do in place sort for qualities

class flexget.utils.qualities.Quality(value, name, all_of=None, none_of=None)

Bases: object

matches(text)

Test if quality matches to text.

Parameters:text (string) – data te be tested against
Returns:tuple (matches, remaining text without quality data)
flexget.utils.qualities.all()

Return all Qualities in order of best to worst

flexget.utils.qualities.common_name(name)

Return common name for name (case insensitive).

Parameters:name (string) – Name to be converted in the common form.
Returns:common name, eg. 1280x720, 720 and 720p will all return 720p
Return type:string
flexget.utils.qualities.get(name, default=None)

Return Quality object for :name: (case insensitive) :param name: Quality name :return: Found Quality / UNKNOWN or default if given and nothing was found.

flexget.utils.qualities.max()

Return highest known Quality.

flexget.utils.qualities.min()

Return lowest known Quality excluding unknown.

flexget.utils.qualities.parse_quality(title)

Find the highest know quality in a given string :title:

Returns:Quality object or False
flexget.utils.qualities.quality_match(title)

Search best quality from title

Parameters:title (string) – text to search from
Returns:tuple (Quality which can be unknown, remaining title without quality)
flexget.utils.qualities.value(name)
Parameters:name (str) – case insensitive quality name
Returns:Return value of quality with given name or 0 if unknown

requests Module

class flexget.utils.requests.Session(**kwargs)

Bases: requests.sessions.Session

Subclass of requests Session class which defines some of our own defaults, records unresponsive sites, and raises errors by default.

add_cookiejar(cookiejar)

Add a CookieJar whose cookies will apply to this session. If the session already has a cookiejar set, the cookies will be merged together.

Parameters:cookiejar – CookieJar instance to add to the session.
request(method, url, *args, **kwargs)

Does a request, but raises Timeout immediately if site is known to timeout, and records sites that timeout. Also raises errors getting the content by default.

set_domain_delay(domain, delay)

Registers a minimum interval between requests to domain

Parameters:
  • domain – The domain to set the interval on
  • delay – The amount of time between requests, can be a timedelta or string like ‘3 seconds’
flexget.utils.requests.cookies_for_url(url, cookiejar)

Returns a dict containing the cookies from cookiejar that apply to given url.

Parameters:
  • url (str) – The url that cookies should apply to.
  • cookiejar (CookieJar) – The CookieJar instance to get cookies from.
Returns:

A dict with cookies that apply to given url.

Return type:

dict

flexget.utils.requests.get(url, **kwargs)

Sends a GET request. Returns Response object.

Parameters:
  • url – URL for the new Request object.
  • **kwargs

    Optional arguments that request takes.

flexget.utils.requests.is_unresponsive(url)

Checks if host of given url has timed out within WAIT_TIME

Parameters:url – The url to check
Returns:True if the host has timed out within WAIT_TIME
Return type:bool
flexget.utils.requests.post(url, data=None, **kwargs)

Sends a POST request. Returns Response object.

Parameters:
  • url – URL for the new Request object.
  • data – (optional) Dictionary or bytes to send in the body of the Request.
  • **kwargs

    Optional arguments that request takes.

flexget.utils.requests.request(method, url, **kwargs)
flexget.utils.requests.set_unresponsive(url)

Marks the host of a given url as unresponsive

Parameters:url – The url that timed out

search Module

Common tools used by plugins implementing search plugin api

class flexget.utils.search.AnyComparator

Bases: object

Comparator that does no comparison. Used to return all results from a search plugin without filtering.

matches(other=None)
ratio()
search_string()

Return a cleaned string based on seq1 that can be used for searching.

set_seq1(a)
set_seq2(b)
class flexget.utils.search.MovieComparator

Bases: flexget.utils.search.StringComparator

Compares two strings for similarity based on extracted movie title, year and quality.

matches(other=None)

Compare the two strings, return True if they appear to be the same movie.

Parameters:other – String to compare against. If not specified, last specified string will be used.
Returns:True if match is close enough.
search_string()

Return a cleaned string based on seq1 that can be used for searching.

set_seq1(a)

Set first string for comparison.

set_seq2(b)

Set second string for comparison.

class flexget.utils.search.StringComparator(cutoff=0.9, cleaner=<function clean_symbols at 0x3c12578>)

Bases: difflib.SequenceMatcher, object

Compares two strings for similarity. Runs a cleaner function on strings before comparison. Cutoff similarity is configurable.

matches(other=None)

Compare the two strings, return True if match is close enough.

Parameters:other – String to compare against. If not specified, last specified string will be used.
Returns:True if match is close enough.
search_string()

Return a cleaned string based on seq1 that can be used for searching.

set_seq1(a)

Set first string for comparison.

set_seq2(b)

Set second string for comparison.

flexget.utils.search.clean_symbols(text)

Replaces common symbols with spaces. Also normalize unicode strings in decomposed form.

flexget.utils.search.clean_title(title)

Removes common codec, sound keywords, and special characters info from titles to facilitate loose title comparison.

flexget.utils.search.torrent_availability(seeds, leeches)

Returns a rating based on seeds and leeches for a given torrent.

Parameters:
  • seeds – Number of seeds on the torrent
  • leeches – Number of leeches on the torrent
Returns:

A numeric rating

simple_persistence Module

NOTE:

Avoid using this module on your own or in plugins, this was originally made for 0.9 -> 1.0 transition.

You can safely use feed.simple_persistence and manager.persist, if we implement something better we can replace underlying mechanism in single point (and provide transparent switch).

class flexget.utils.simple_persistence.SimpleFeedPersistence(feed)

Bases: flexget.utils.simple_persistence.SimplePersistence

feedname
plugin
session
class flexget.utils.simple_persistence.SimpleKeyValue(feed, plugin, key, value)

Bases: sqlalchemy.ext.declarative.Base

Declarative

added
feed
id
key
plugin
value
class flexget.utils.simple_persistence.SimplePersistence(plugin, session=None)

Bases: UserDict.DictMixin

keys()

soup Module

flexget.utils.soup.get_soup(obj)

sqlalchemy_utils Module

Miscellaneous SQLAlchemy helpers.

flexget.utils.sqlalchemy_utils.drop_tables(names, session)

Takes a list of table names and drops them from the database if they exist.

flexget.utils.sqlalchemy_utils.get_index_by_name(table, name)

Find declaratively defined index from table by name

Parameters:
  • table – Table object
  • name (string) – Name of the index to get
Returns:

Index object

flexget.utils.sqlalchemy_utils.table_add_column(table, name, col_type, session, default=None)

Adds a column to a table

Warning

Uses raw statements, probably needs to be changed in order to work on other databases besides SQLite

Parameters:
  • table (string) – Table to add column to (can be name or schema)
  • name (string) – Name of new column to add
  • col_type – The sqlalchemy column type to add
  • session (Session) – SQLAlchemy Session to do the alteration
  • default – Default value for the created column (optional)
flexget.utils.sqlalchemy_utils.table_columns(table, session)
Parameters:
  • table (string) – Name of table or table schema
  • session (Session) – SQLAlchemy Session
Returns:

List of column names in the table or empty list

flexget.utils.sqlalchemy_utils.table_exists(name, session)

Use SQLAlchemy reflect to check table existences.

Parameters:
  • name (string) – Table name to check
  • session (Session) – Session to use
Returns:

True if table exists, False otherwise

Return type:

bool

flexget.utils.sqlalchemy_utils.table_schema(name, session)
Returns:Table schema using SQLAlchemy reflect as it currently exists in the db
Return type:Table

template Module

exception flexget.utils.template.RenderError

Bases: exceptions.Exception

Error raised when there is a problem with jinja rendering.

flexget.utils.template.filter_format_number(val, places=None, grouping=True)

Formats a number according to the user’s locale.

flexget.utils.template.filter_formatdate(val, format)

Returns a string representation of a datetime object according to format string.

flexget.utils.template.filter_pad(val, width, fillchar='0')

Pads a number or string with fillchar to the specified width.

flexget.utils.template.filter_parsedate(val)

Attempts to parse a date according to the rules in RFC 2822

flexget.utils.template.filter_pathbase(val)

Base name of a path.

flexget.utils.template.filter_pathdir(val)

Directory containing the given path.

flexget.utils.template.filter_pathext(val)

Extension of a path (including the ‘.’).

flexget.utils.template.filter_pathname(val)

Base name of a path, without its extension.

flexget.utils.template.filter_pathscrub(val, ascii=False, windows=None)

Replace problematic characters in a path.

flexget.utils.template.filter_re_replace(val, pattern, repl)

Perform a regexp replacement on the given string.

Perform a search for given regexp pattern, return the matching portion of the text.

flexget.utils.template.get_template(templatename, pluginname=None)
flexget.utils.template.make_environment(manager)

Create our environment and add our custom filters

flexget.utils.template.render_from_entry(template_string, entry)

Renders a Template or template string with an Entry as its context.

flexget.utils.template.render_from_feed(template, feed)

Renders a Template with an feed as its context.

tools Module

Contains miscellaneous helpers

exception flexget.utils.tools.MergeException(value)

Bases: exceptions.Exception

class flexget.utils.tools.ReList(*args, **kwargs)

Bases: list

A list that stores regexps.

You can add compiled or uncompiled regexps to the list. It will always return the compiled version. It will compile the text regexps on demand when first accessed.

flags = 34
class flexget.utils.tools.SmartRedirectHandler

Bases: urllib2.HTTPRedirectHandler

http_error_301(req, fp, code, msg, headers)
http_error_302(req, fp, code, msg, headers)
flexget.utils.tools.console(text)

Print to console safely.

flexget.utils.tools.convert_bytes(bytes)

Returns given bytes as prettified string.

flexget.utils.tools.decode_html(value)
Parameters:value (string) – String to be html-decoded
Returns:Html decoded string
flexget.utils.tools.encode_html(unicode_data, encoding='ascii')

Encode unicode_data for use as XML or HTML, with characters outside of the encoding converted to XML numeric character references.

flexget.utils.tools.make_valid_path(path, windows=None)

Removes invalid characters from windows pathnames

flexget.utils.tools.merge_dict_from_to(d1, d2)

Merges dictionary d1 into dictionary d2. d1 will remain in original form.

flexget.utils.tools.parse_timedelta(value)
flexget.utils.tools.sanitize(value, logger=None)
flexget.utils.tools.sanitize_dict(d, logger=None)

Makes dictionary d contain only yaml.safe_dump compatible elements. On other words, remove all non standard types from dictionary.

flexget.utils.tools.sanitize_list(content, logger=None)
flexget.utils.tools.str_to_boolean(string)
flexget.utils.tools.strip_html(text)

Tries to strip all HTML tags from text. If unsuccessful returns original text.

flexget.utils.tools.urlopener(url_or_request, log, **kwargs)

Utility function for pulling back a url, with a retry of 3 times, increasing the timeout, etc. Re-raises any errors as URLError.

Warning

This is being replaced by requests library. flexget.utils.requests should be used going forward.

Parameters:
  • url_or_request (str) – URL or Request object to get.
  • log – Logger to log debug info and errors to
  • kwargs – Keyword arguments to be passed to urlopen
Returns:

The file-like object returned by urlopen