pyoko package

Subpackages

• pyoko.db package
  • Submodules
  • pyoko.db.queryset module
  • pyoko.db.connection module
  • pyoko.db.schema_update module
  • Module contents
• pyoko.lib package
  • Submodules
  • pyoko.lib.py2map module
  • pyoko.lib.utils module
  • Module contents

Submodules

pyoko.conf module

class pyoko.conf.Settings

Bases: object

pyoko.exceptions module

exception pyoko.exceptions.IntegrityError

Bases: pyoko.exceptions.PyokoError

raised on unique/unique_together mismatches

exception pyoko.exceptions.MultipleObjectsReturned

Bases: pyoko.exceptions.PyokoError

The query returned multiple objects when only one was expected.

exception pyoko.exceptions.NoSuchObjectError

Bases: pyoko.exceptions.PyokoError

exception pyoko.exceptions.NotCompatible

Bases: pyoko.exceptions.PyokoError

Incorrect usage of method / function

exception pyoko.exceptions.ObjectDoesNotExist

Bases: pyoko.exceptions.PyokoError

exception pyoko.exceptions.PyokoError

Bases: exceptions.Exception

exception pyoko.exceptions.ValidationError

Bases: pyoko.exceptions.PyokoError

pyoko.fields module

class pyoko.fields.BaseField(title='', default=None, required=True, index=True, type=None, store=False, choices=None, order=None, unique=False, help_text=None, **kwargs)

Bases: object

clean_value(val)

creation_counter = 4

default_value = None

validate(val)

class pyoko.fields.Boolean(title='', default=None, required=True, index=True, type=None, store=False, choices=None, order=None, unique=False, help_text=None, **kwargs)

Bases: pyoko.fields.BaseField

clean_value(val)

solr_type = 'boolean'

class pyoko.fields.Date(*args, **kwargs)

Bases: pyoko.fields.BaseField

clean_value(val)

solr_type = 'date'

class pyoko.fields.DateTime(*args, **kwargs)

Bases: pyoko.fields.BaseField

clean_value(val)

solr_type = 'date'

class pyoko.fields.File(*args, **kwargs)

Bases: pyoko.fields.BaseField

clean_value(val)

val = :param dict val: {“content”:”“, “name”:”“, “ext”:”“, “type”:”“} :return:

file_manager

solr_type = 'file'

class pyoko.fields.Float(title='', default=None, required=True, index=True, type=None, store=False, choices=None, order=None, unique=False, help_text=None, **kwargs)

Bases: pyoko.fields.BaseField

Numeric field that holds float data.

clean_value(val)

solr_type = 'float'

class pyoko.fields.Id(*arg, **kwargs)

Bases: pyoko.fields.BaseField

clean_value(val)

solr_type = 'string'

class pyoko.fields.Integer(title='', default=None, required=True, index=True, type=None, store=False, choices=None, order=None, unique=False, help_text=None, **kwargs)

Bases: pyoko.fields.BaseField

clean_value(val)

solr_type = 'int'

class pyoko.fields.String(title='', default=None, required=True, index=True, type=None, store=False, choices=None, order=None, unique=False, help_text=None, **kwargs)

Bases: pyoko.fields.BaseField

solr_type = 'string'

class pyoko.fields.Text(title='', default=None, required=True, index=True, type=None, store=False, choices=None, order=None, unique=False, help_text=None, **kwargs)

Bases: pyoko.fields.BaseField

Text field.

solr_type = 'text_general'

class pyoko.fields.TimeStamp(*args, **kwargs)

Bases: pyoko.fields.BaseField

clean_value(val)

solr_type = 'date'

pyoko.listnode module

This module holds the ListNode implementation of Pyoko Models.

ListNode’s are used to model ManyToMany relations and other list like data types on a Model.

class pyoko.listnode.ListNode(**kwargs)

Bases: pyoko.node.Node

ListNode’s are used to store list of field sets. Their DB representation look like list of dicts:

class Student(Model):
    class Lectures(ListNode):
        name = field.String()
        code = field.String(required=False)

st = Student()
st.Lectures(name="Math101", code='M1')
st.Lectures(name="Math102", code='M2')
st.clean_value()
{
    'deleted': False,
    'timestamp': None
    'lectures': [
        {'code': 'M1', 'name': 'Math101'},
        {'code': 'M2', 'name': 'Math102'},
    ]
}

Notes

• Currently we disregard the ordering of ListNode items.
• “reverse_name” dose not supported on linked models.

__call__(**kwargs)

Stores created instance in node_stack and returns it’s reference to callee

__delattr__

x.__delattr__(‘name’) <==> del x.name

__delitem__(obj, sync=True)

Allow usage of “del” statement on ListNodes with bracket notation.

Parameters:obj – ListNode item or relation key. Raises:TypeError – If it’s called on a ListNode item (intstead of ListNode’s itself)

__format__()

default object formatter

__getattribute__

x.__getattribute__(‘name’) <==> x.name

__hash__

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__repr__()

This works for two different object:

• Main ListNode object
• Items of the ListNode (like instance of a class) which created while iterating on main ListNode object

Returns:String representation of object.

__sizeof__() → int

size of object in memory, in bytes

add(**kwargs)

Stores node data without creating an instance of it. This is more efficient if node instance is not required.

Parameters:kwargs – attributes of the ListNode

clean_value()

Populates json serialization ready data. This is the method used to serialize and store the object data in to DB

Returns:List of dicts.

clear()

Clear outs the list node.

Raises:TypeError – If it’s called on a ListNode item (intstead of ListNode’s itself)

classmethod get_field(field_name)

get_humane_value(name)

Returns a human readble/meaningful value for the field

Parameters:name (str) – Model field name Returns:Human readble field value

classmethod get_link(**kw)

Get first item of get_links() method

classmethod get_links(**kw)

Linked models from this model

Keyword Arguments:  

• by items of _linked_models (filter) –
• startswith – Bool, use startswith for filtering instead of exact matching. Only works for filtering over one field.

Returns:

Link list

get_verbose_name()

Returns:value of title attribute or verbose name of class

objects

pre_add()

A hook for doing things before adding new listnode item to the stack

remove()

Removes an item from ListNode.

Raises:TypeError – If it’s called on container ListNode (intstead of ListNode’s item)

Note

Parent object should be explicitly saved.

setattr(key, val)

setattrs(**kwargs)

pyoko.manage module

command line management interface

class pyoko.manage.BaseDumpHandler(models, batch_size, threads, per_model=False, output_path='', remove_dumped=False)

Bases: pyoko.manage.BaseThreadedCommand

The base class for different implementations of data dump handlers.

EXTENSION = 'dump'

dump_data()

dump_model_data(mdl)

handle_data(bucket, key, value)

multi_file

post_dump_hook(bucket)

post_handle_data_hook(bucket, key, value)

pre_dump_hook(bucket)

pre_handle_data_hook(bucket, key, value)

single_file

write(data)

class pyoko.manage.BaseThreadedCommand

Bases: object

do_with_submit(func, iterables, *args, **kwargs)

Parameters:

• func – function name to execute
• iterables – iterables list to execute with given function

Returns:

find_models()

class pyoko.manage.CSVDumpHandler(models, batch_size, threads, per_model=False, output_path='', remove_dumped=False)

Bases: pyoko.manage.BaseDumpHandler

This is the default format. Writes one record per line. Since it bypasses the JSON encoding/decoding, it’s much faster and memory efficient than others.

EXTENSION = 'csv'

handle_data(bucket, key, value)

class pyoko.manage.Command(manager=None, **kwargs)

Bases: object

Command object is a thin wrapper around Python’s powerful argparse module. Holds the given command line parameters in self.manager.args

Variables:

• CMD_NAME – name of your command
• HELP – help texts starts with “R|” will be parsed as raw text
• PARAMS –

  A dictionary list with following possible values.

  • name: name of parameter
  • help: help text for parameter. Parsed as raw if starts with “R|”
  • required: Optional. Set True if this is a required parameter.
  • default: Optional. Define a default value for the parameter
  • action: ‘store_true’ see the official argparse documentation for more info

run()

This is where the things are done. You should override this method in your command class.

class pyoko.manage.CommandRegistry(name, bases, attrs)

Bases: type

classmethod add_command(command_model)

classmethod get_commands()

CommandRegistry.registry [{}]

class pyoko.manage.DumpData(manager=None, **kwargs)

Bases: pyoko.manage.Command, pyoko.manage.BaseThreadedCommand

CMD_NAME = 'dump_data'

DUMP_HANDLERS = {'csv': <class 'pyoko.manage.CSVDumpHandler'>, 'json': <class 'pyoko.manage.JSONDumpHandler'>, 'json_tree': <class 'pyoko.manage.TreeDumpHandler'>, 'pretty': <class 'pyoko.manage.PrettyDumpHandler'>}

HELP = 'Dumps all data to stdout or to given file'

DumpData.PARAMS [{}]

run()

This is where the things are done. You should override this method in your command class.

class pyoko.manage.FindDuplicateKeys(manager=None, **kwargs)

Bases: pyoko.manage.Command

CMD_NAME = '_find_dups'

HELP = 'finds duplicate keys, to help debugging'

run()

This is where the things are done. You should override this method in your command class.

class pyoko.manage.FlushDB(manager=None, **kwargs)

Bases: pyoko.manage.Command, pyoko.manage.BaseThreadedCommand

CMD_NAME = 'flush_model'

HELP = 'REALLY DELETES the contents of models'

FlushDB.PARAMS [{}]

flush_model(mdl)

run()

This is where the things are done. You should override this method in your command class.

sync_flush_model(mdl)

class pyoko.manage.GenerateDiagrams(manager=None, **kwargs)

Bases: pyoko.manage.Command

CMD_NAME = 'generate_diagrams'

HELP = 'Generate PlantUML diagrams from the models.'

GenerateDiagrams.PARAMS [{}]

SPLIT_APP = 'app'

SPLIT_CHOICES = ('no', 'app', 'model')

SPLIT_MODEL = 'model'

SPLIT_NO = 'no'

run()

This is where the things are done. You should override this method in your command class.

class pyoko.manage.JSONDumpHandler(models, batch_size, threads, per_model=False, output_path='', remove_dumped=False)

Bases: pyoko.manage.BaseDumpHandler

Writes each line as a separate JSON document. Unlike “json_tree”, memory usage does not increase with the number of records.

EXTENSION = 'json'

handle_data(bucket, key, value)

class pyoko.manage.LoadData(manager=None, **kwargs)

Bases: pyoko.manage.Command, pyoko.manage.BaseThreadedCommand

Loads previously dumped data into DB.

CHOICES = ('csv', 'json', 'json_tree', 'pretty')

CMD_NAME = 'load_data'

CSV = 'csv'

HELP = 'Reads JSON data from given file and populates models'

JSON = 'json'

LoadData.PARAMS [{}]

PRETTY = 'pretty'

TREE = 'json_tree'

prepare_buckets()

loads buckets to bucket cache.

read_each_file(file)

read_file(file_path)

read_json_per_line(file)

read_per_line(file)

read_whole_file(file)

run()

This is where the things are done. You should override this method in your command class.

save_obj(bucket_name, key, val)

class pyoko.manage.ManagementCommands(args=None)

Bases: object

All management commands executed by this class. You can create your own commands by extending Command class

daemonize()

parse_args(args)

class pyoko.manage.PrettyDumpHandler(*args, **kwargs)

Bases: pyoko.manage.TreeDumpHandler

DO NOT use on big DBs. Formatted version of json_tree.

post_dump_hook(bucket)

class pyoko.manage.ReIndex(manager=None, **kwargs)

Bases: pyoko.manage.Command, pyoko.manage.BaseThreadedCommand

CMD_NAME = 'reindex'

HELP = 'Re-indexes model objects'

ReIndex.PARAMS [{}]

reindex_model(mdl)

run()

This is where the things are done. You should override this method in your command class.

class pyoko.manage.SchemaUpdate(manager=None, **kwargs)

Bases: pyoko.manage.Command, pyoko.manage.BaseThreadedCommand

CMD_NAME = 'migrate'

HELP = 'Creates/Updates SOLR schemas for given model(s)'

SchemaUpdate.PARAMS [{}]

run()

This is where the things are done. You should override this method in your command class.

class pyoko.manage.Shell(manager=None, **kwargs)

Bases: pyoko.manage.Command

CMD_NAME = 'shell'

HELP = 'Run IPython shell'

Shell.PARAMS [{}]

run()

This is where the things are done. You should override this method in your command class.

class pyoko.manage.SmartFormatter(prog, indent_increment=2, max_help_position=24, width=None)

Bases: argparse.HelpFormatter

class pyoko.manage.TestGetKeys(manager=None, **kwargs)

Bases: pyoko.manage.Command

CMD_NAME = '_test_get_keys'

HELP = 'tests the correctness of the bucket.get_keys()'

run()

This is where the things are done. You should override this method in your command class.

class pyoko.manage.TreeDumpHandler(*args, **kwargs)

Bases: pyoko.manage.BaseDumpHandler

DO NOT use on big DBs. Writes whole dump as a big JSON object.

EXTENSION = 'json'

handle_data(bucket, key, value)

post_dump_hook(bucket)

pyoko.model module

This module holds the pyoko’s Model object

class pyoko.model.LinkProxy(link_to, one_to_one=False, verbose_name=None, reverse_name=None, null=False, unique=False)

Bases: object

Proxy object for “self” referencing model relations .. rubric:: Example

class Unit(Model):
    name = field.String("Name")
    parent = LinkProxy('Unit', verbose_name='Upper unit', reverse_name='sub_units')

__delattr__

x.__delattr__(‘name’) <==> del x.name

__format__()

default object formatter

__getattribute__

x.__getattribute__(‘name’) <==> x.name

__hash__

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__repr__

__setattr__

x.__setattr__(‘name’, value) <==> x.name = value

__sizeof__() → int

size of object in memory, in bytes

__str__

class pyoko.model.Model(context=None, **kwargs)

Bases: pyoko.node.Node

This is base class for any model object.

Field instances are used as model attributes to represent values.

class Permission(Model):
    name = field.String("Name")
    code = field.String("Code Name")

    def __unicode__(self):
        return "%s %s" % (self.name, self.code)

Models may have inner classes to represent ManyToMany relations, inner data nodes or lists.

Notes

• “reverse_name” does not supported on links from ListNode’s.

__delattr__

x.__delattr__(‘name’) <==> del x.name

__eq__(other)

Equivalence of two model instance depends on uniformity of their self._data and self.key.

__format__()

default object formatter

__getattribute__

x.__getattribute__(‘name’) <==> x.name

__ne__(other)

İnequality of two model instance depends on uniformity of their self._data and self.key.

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__sizeof__() → int

size of object in memory, in bytes

blocking_delete(meta=None, index_fields=None)

Deletes and waits till the backend properly update indexes for just deleted object. meta (dict): JSON serializable meta data for logging of save operation.

{‘lorem’: ‘ipsum’, ‘dolar’: 5}

index_fields (list): Tuple list for indexing keys in riak (with ‘bin’ or ‘int’).

bin is used for string fields, int is used for integer fields. [(‘lorem’,’bin’),(‘dolar’,’int’)]

blocking_save(query_dict=None, meta=None, index_fields=None)

Saves object to DB. Waits till the backend properly indexes the new object.

Parameters:

• query_dict (dict) – contains keys - values of the model fields
• meta (dict) – JSON serializable meta data for logging of save operation. {‘lorem’: ‘ipsum’, ‘dolar’: 5}
• index_fields (list) – Tuple list for indexing keys in riak (with ‘bin’ or ‘int’). bin is used for string fields, int is used for integer fields. [(‘lorem’,’bin’),(‘dolar’,’int’)]

Returns:

Model instance.

changed_fields(from_db=False)

Parameters:from_db (bool) – Check changes against actual db data Returns:List of fields names which their values changed. Return type:list

clean_value()

generates a json serializable representation of the model data :rtype: dict :return: riak ready python dict

delete(dry=False, meta=None, index_fields=None)

Sets the objects “deleted” field to True and, current time to “deleted_at” fields then saves it to DB.

Parameters:

• dry (bool) – False. Do not execute the actual deletion.
• list what will be deleted as a result of relations. (Just) –
• meta (dict) – JSON serializable meta data for logging of save operation. {‘lorem’: ‘ipsum’, ‘dolar’: 5}
• index_fields (list) – Tuple list for secondary indexing keys in riak (with ‘bin’ or ‘int’). bin is used for string fields, int is used for integer fields. [(‘lorem’,’bin’),(‘dolar’,’int’)]

Returns:

Tuple. (results [], errors [])

exist

Used to check if a relation is exist or a model instance is saved to DB or not.

Returns:True if this model instance stored in DB and has a key and False otherwise.

get_choices_for(field)

Get the choices for the given fields.

Parameters:field (str) – Name of field. Returns:List of tuples. [(name, value),…]

classmethod get_field(field_name)

get_humane_value(name)

Returns a human readble/meaningful value for the field

Parameters:name (str) – Model field name Returns:Human readble field value

classmethod get_link(**kw)

Get first item of get_links() method

classmethod get_links(**kw)

Linked models from this model

Keyword Arguments:  

• by items of _linked_models (filter) –
• startswith – Bool, use startswith for filtering instead of exact matching. Only works for filtering over one field.

Returns:

Link list

get_unpermitted_fields()

Gives unpermitted fields for current context/user.

Returns:List of unpermitted field names.

get_verbose_name()

Returns:Verbose name of the model instance

is_changed(field, from_db=False)

Parameters:

• field (string) – Field name.
• from_db (bool) – Check changes against actual db data

Returns:

True if given fields value is changed.

Return type:

bool

is_in_db()

Deprecated:

Use “exist” property instead.

objects

alias of pyoko.db.queryset.QuerySet

post_creation()

Called after object’s creation (first save). Can be overriden to do things that should be done after object saved to DB.

post_delete()

Called after object deletion. Can be overriden to do things that should be done after object is marked deleted.

post_save()

Called after object save. Can be overriden to do things that should be done after object saved to DB.

pre_creation()

Called before object’s creation (first save). Can be overriden to do things that should be done before object saved to DB.

pre_delete()

Called before object deletion. Can be overriden to do things that should be done before object is marked deleted.

pre_save()

Called before object save. Can be overriden to do things that should be done just before object saved to DB.

prnt()

Prints DB data representation of the object.

reload()

Reloads current instance from DB store

static row_level_access(context, objects)

Can be used to implement context-aware implicit filtering. You can define your query filters in here to enforce row level access control.

If defined, will be called at queryset initialization step and it’s return value used as Model.objects.

Parameters:

• context – An object that contain required user attributes and permissions.
• objects (Queryset) – QuerySet object.

Examples

Returns:Queryset object.

save(internal=False, meta=None, index_fields=None)

Save’s object to DB.

Do not override this method, use pre_save and post_save methods.

Parameters:

• internal (bool) – True if called within model. Used to prevent unneccessary calls to pre_save and post_save methods.
• meta (dict) – JSON serializable meta data for logging of save operation. {‘lorem’: ‘ipsum’, ‘dolar’: 5}
• index_fields (list) – Tuple list for indexing keys in riak (with ‘bin’ or ‘int’). bin is used for string fields, int is used for integer fields. [(‘lorem’,’bin’),(‘dolar’,’int’)]

Returns:

Saved model instance.

set_data(data, from_db=False)

Fills the object’s fields with given data dict. Internally calls the self._load_data() method.

Parameters:

• data (dict) – Data to fill object’s fields.
• from_db (bool) – if data coming from db then we will
• related field type's _load_data method (use) –

Returns:

Self. Returns objects itself for chainability.

setattr(key, val)

setattrs(**kwargs)

pyoko.modelmeta module

class pyoko.modelmeta.ModelMeta(name, bases, attrs)

Bases: type

Metaclass that process model classes.

static process_attributes_of_node(attrs, node_name, class_type)

prepare the model fields, nodes and relations

Parameters:

• node_name (str) – name of the node we are currently processing
• attrs (dict) – attribute dict
• class_type (str) – Type of class. Can be one of these: ‘ListNode’, ‘Model’, ‘Node’

static process_models(attrs, base_model_class)

Attach default fields and meta options to models

static process_objects(kls)

Applies default Meta properties.

pyoko.node module

class pyoko.node.FakeContext

Bases: object

this fake context object can be used to use ACL limited models from shell

has_permission(perm)

class pyoko.node.LazyModel(wrapped, null, verbose_name)

Bases: Proxy

exist

get_verbose_name()

key = None

null = False

verbose_name = None

class pyoko.node.Node(**kwargs)

Bases: object

We store node classes in _nodes[] attribute at ModelMeta, then replace them with their instances at _instantiate_nodes()

Likewise we store linked models in _linked_models[]

Since fields are defined as descriptors, they can access to instance they called from but to access their methods and attributes, we’re copying fields themself into self._fields[] attribute. So, we get values of fields from self._field_values[] and access to fields themselves from self._fields[]

clean_value()

generates a json serializable representation of the model data :rtype: dict :return: riak ready python dict

classmethod get_field(field_name)

get_humane_value(name)

Returns a human readble/meaningful value for the field

Parameters:name (str) – Model field name Returns:Human readble field value

classmethod get_link(**kw)

Get first item of get_links() method

classmethod get_links(**kw)

Linked models from this model

Keyword Arguments:  

• by items of _linked_models (filter) –
• startswith – Bool, use startswith for filtering instead of exact matching. Only works for filtering over one field.

Returns:

Link list

get_verbose_name()

Returns:value of title attribute or verbose name of class

setattr(key, val)

setattrs(**kwargs)

pyoko.registry module

class pyoko.registry.FakeContext

Bases: object

has_permission(perm)

class pyoko.registry.Registry

Bases: object

get_apps()

get_base_models()

get_model(model_name)

get_models_by_apps()

get_models_of_app(app_name)

register_model(mdl)

Module contents