python 分享一个用SSDB 做缓存的类

分享一个用SSDB 做缓存的类,使用python sdk,可以作为一个服务跑起来。

python 分享一个用SSDB 做缓存的类

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
__doc__ = """
ssdb_cache has a dictionary like interface and a SSDB(http://ssdb.io/) backend
It uses pickle to store Python objects and strings
Multithreading is supported
"""
import datetime
# pip install pyssdb --upgrade
import pyssdb
try:
    import cPickle as pickle
except ImportError:
    import pickle

class CACHE:
    """Stores and retrieves persistent data through a dict-like interface
    Data is stored on disk using SSDB

    cache_name/filename: 
        The name of Hashmap, use this name "filename" to be compatible with PersistentDict in webscraping library
    expires: 
        A timedelta object of how old data can be before expires. By default is set to None to disable. 
    host:
        The SSDB server host
    port:
        The SSDB server port
    """
    def __init__(self, cache_name=None, filename=None, expires=None, host='127.0.0.1', port=8888):
        """initialize a new CACHE
        """
        self.cache_name = filename or cache_name or 'cache'
        self.expires = expires
        self.ssdb_host = host
        self.ssdb_port = port
        self.ssdb_client = pyssdb.Client(host=self.ssdb_host, port=self.ssdb_port)

    def __copy__(self):
        """make a copy of current cache settings
        """
        return CACHE(cache_name=self.cache_name, expires=self.expires, host=self.ssdb_host, port=self.ssdb_port)

    def __contains__(self, key):
        """check the database to see if a key exists
        """
        return self.ssdb_client.hexists(self.cache_name, key)

    def __getitem__(self, key):
        """return the value of the specified key or raise KeyError if not found
        """
        value_meta_update = self.ssdb_client.hget(self.cache_name, key)
        if value_meta_update is not None:
            value_meta_update = self.deserialize(value_meta_update)
            if self.is_fresh(value_meta_update.get('updated')):
                return value_meta_update.get('value')
            else:
                raise KeyError("Key `%s' is stale" % key)
        else:
            raise KeyError("Key `%s' does not exist" % key)

    def __delitem__(self, key):
        """remove the specifed value from the database
        """
        self.ssdb_client.hdel(self.cache_name, key)

    def __setitem__(self, key, value):
        """set the value of the specified key
        """
        value_meta_update = {'value': value, 'meta': None, 'updated': datetime.datetime.now()}
        self.ssdb_client.hset(self.cache_name, key, self.serialize(value_meta_update))
        
    def __len__(self):
        """get the number of elements in CACHE
        """
        return self.ssdb_client.hsize(self.cache_name)        

    def serialize(self, value):
        """convert object to a pickled string to save in the db
        """
        return pickle.dumps(value, protocol=pickle.HIGHEST_PROTOCOL)
    
    def deserialize(self, value):
        """convert pickled string from database back into an object
        """
        return pickle.loads(value)

    def get(self, key, default=None):
        """Get data at key and return default if not defined
        """
        data = default
        if key:
            value_meta_update = self.ssdb_client.hget(self.cache_name, key)
            if value_meta_update is not None:
                return self.deserialize(value_meta_update)
        return data

    def meta(self, key, value=None):
        """Get / set meta for this value

        if value is passed then set the meta attribute for this key
        if not then get the existing meta data for this key
        """
        if value is None:
            # want to get meta
            value_meta_update = self.ssdb_client.hget(self.cache_name, key)
            if value_meta_update is not None:
                return self.deserialize(value_meta_update).get('meta')
            else:
                raise KeyError("Key `%s' does not exist" % key)
        else:
            # want to set meta
            value_meta_update = self.ssdb_client.hget(self.cache_name, key)
            if value_meta_update is not None:
                value_meta_update = self.deserialize(value_meta_update)
                value_meta_update['meta'] = value
                value_meta_update['updated'] = datetime.datetime.now()
                self.ssdb_client.hset(self.cache_name, key, self.serialize(value_meta_update))

    def clear(self):
        """Clear all cached data in this collecion
        """
        self.ssdb_client.hclear(self.cache_name)
        
    def is_fresh(self, t):
        """returns whether this datetime has expired
        """
        return self.expires is None or datetime.datetime.now() - t < self.expires 
    
if __name__ == '__main__':
    cache = CACHE(cache_name='test')
    print 'Name' in cache
    cache['Name'] = 'Peng Qi'
    print cache['Name']
    print cache.get('Name')
    cache.meta('Name', {'Age': 29})
    print cache.meta('Name')

本文网址: https://pylist.com/topic/105.html 转摘请注明来源

Suggested Topics

ssdb python 接口提速

SSDB 是个新兴的数据库,其数据库的特点简单,性能高效,有好多python 接口,个人比较后选择一个最理想的,但还有提速空间,这里仅作经验分享。...

ssdb 全文搜索的实现

ssdb 作为key-value 数据库,底层没有提供全文搜索的功能,只能在应用层作检索。...

python 正确计算大文件md5 值

python 计算文件的md5值很方便,但如果只是简单的把文件都入到内存中,大文件会导致问题,一般采用切片的方式分段计算,下面的几个函数可以很好的解决这个问题。...

python 处理命令行参数

Python 完全支持创建在命令行运行的程序,也支持通过命令行参数和短长样式来指定各种选项。...

在SAE Python上开启gzip的方法

开启 gzip 的作用自不必说,可以省很多流出带宽,可以省很多云豆。昨天这个博客的云豆消耗,其中流出带宽就占九成多,开启后就会只占五成多。...

python编程中常用的12种基础知识总结

python编程中常用的12种基础知识总结:正则表达式替换,遍历目录方法,列表按列排序、去重,字典排序,字典、列表、字符串互转,时间对象操作,命令行参数解析(getopt),print 格式化输出,进制转换,Python调用系统命令或者脚本,Python 读写文件。...

python 标准库读写CSV 文件

CSV 是(Comma Separated Values 逗号分隔值)的英文缩写,通常都是纯文本文件。这里介绍使用python 标准库读写csv 文件的方法。...

Leave a Comment