{"_id":"553d394e9660ba0d00569d0a","user":"543466ea0e8e2b0e00341818","__v":0,"category":{"_id":"553d394d9660ba0d00569d09","pages":["553d394e9660ba0d00569d0a","553d394e9660ba0d00569d0b","553d394e9660ba0d00569d0c","553d394e9660ba0d00569d0d","553d394e9660ba0d00569d0e","553d394e9660ba0d00569d0f","553d394e9660ba0d00569d10","553d394e9660ba0d00569d11","553d394e9660ba0d00569d12"],"project":"543467200ef9c00800164ecc","__v":1,"version":"553d394d9660ba0d00569d07","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2014-10-15T18:17:35.811Z","from_sync":false,"order":1,"slug":"extensions","title":"EXTENSIONS"},"version":{"_id":"553d394d9660ba0d00569d07","project":"543467200ef9c00800164ecc","forked_from":"54bedb2fffb79c320030db1f","__v":1,"createdAt":"2015-04-26T19:15:25.245Z","releaseDate":"2015-04-26T19:15:25.245Z","categories":["553d394d9660ba0d00569d08","553d394d9660ba0d00569d09"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"","version_clean":"0.11.2","version":"0.11.2"},"githubsync":"","project":"543467200ef9c00800164ecc","metadata":{"title":"","description":"","image":[]},"updates":[],"next":{"pages":[],"description":""},"createdAt":"2014-10-15T18:21:26.186Z","link_external":false,"link_url":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"auth":"never","params":[],"url":""},"isReference":false,"order":3,"body":"# glim-jobqueue - a redis jobqueue extension for glim\n\nThis is a jobqueue implementation for glim framework. It uses [glim-redis](doc:redis). extension for redis connection. You can use job extension for async processes that consume much more time than a typical request. For instance, it can be done mail queue, push notifications, etc. as jobs.\n\n# Installation\n- First, install [glim-redis](doc:redis) extension\n- Clone the github repo, move job folder to ext folder\n- Remove `.git` directory.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"# install glim-extensions from pip\\npip install glim-extensions\",\n      \"language\": \"shell\"\n    }\n  ]\n}\n[/block]\n# Configuration\n- Append job configuration just after gredis configuration in your config file\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"# app/config/<env>.py\\nconfig = {\\n    'extensions' : {\\n        'gredis' : {\\n            'default' : {\\n                'host' : 'localhost',\\n                'port' : '6379',\\n                'db'   : 0\\n            }\\n        },\\n        'job' : {\\n            'default' : {\\n                'redis'  : 'default',\\n                'jobs'   : 'jobs',    # redis list name for jobs\\n                'failed' : 'failed',  # redis list name for failed jobs\\n            }\\n        },\\n    },\\n    # ...\\n}\",\n      \"language\": \"python\"\n    }\n  ]\n}\n[/block]\n# Initializing Job Extension\nTo initialize job extension, type the following;\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"glim job:init\\n# this will create a job.py on your app folder\",\n      \"language\": \"shell\"\n    }\n  ]\n}\n[/block]\n# Creating Jobs\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"# app/jobs.py\\nfrom glim import Log\\nfrom glim_extensions.job import Job\\n\\nclass HelloJob(Job):\\n    def run(self): # self.data is registered when a Job \\n        if 'author' in self.data.keys():\\n            Log.info('hello %s' % self.data['author'])\\n        else:\\n            Log.info('hello glim!')\",\n      \"language\": \"python\"\n    }\n  ]\n}\n[/block]\n# Producing Jobs\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"from glim_extensions.job import JobQueue\\ndata = {\\n    'author' : 'Aras Can Akin'\\n}\\nJobQueue.push(HelloJob(data)) # returns True or False\\nJobQueue.push(HelloJob)\",\n      \"language\": \"python\"\n    }\n  ]\n}\n[/block]\n# Consuming Jobs\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"glim job:consume --name jobs\\n# output\\n# hello Aras Can Akin\\n# hello\",\n      \"language\": \"shell\"\n    }\n  ]\n}\n[/block]\n# Failed Jobs\nIn jobqueue extension, the failed jobs can be moved to an another redis list by raising exceptions. The `$ glim job:consume` command is looking for jobs to throw exception if a failed job occurs.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"# app/jobs.py\\nfrom glim import Log\\nfrom glim_extensions.job import FailedJobError\\nfrom glim_extensions.job import Job\\n\\nclass HelloJob(Job):\\n    def run(self):\\n        if 'author' in self.data.keys():\\n            Log.info('hello %s' % self.data['author'])\\n        else:\\n            raise FailedJobError()\",\n      \"language\": \"python\"\n    }\n  ]\n}\n[/block]\nIn this example, when you push a job without author, the job will go to failed list in redis\n\n# Roadmap\n- the job system should work for many other message queue services like AWSQ, RabbitMQ, IronMQ, etc.\n- command for flushing job queue\n- add feature to push/pop scheduled jobs","excerpt":"","slug":"job-queue","type":"basic","title":"Job Queue"}
# glim-jobqueue - a redis jobqueue extension for glim This is a jobqueue implementation for glim framework. It uses [glim-redis](doc:redis). extension for redis connection. You can use job extension for async processes that consume much more time than a typical request. For instance, it can be done mail queue, push notifications, etc. as jobs. # Installation - First, install [glim-redis](doc:redis) extension - Clone the github repo, move job folder to ext folder - Remove `.git` directory. [block:code] { "codes": [ { "code": "# install glim-extensions from pip\npip install glim-extensions", "language": "shell" } ] } [/block] # Configuration - Append job configuration just after gredis configuration in your config file [block:code] { "codes": [ { "code": "# app/config/<env>.py\nconfig = {\n 'extensions' : {\n 'gredis' : {\n 'default' : {\n 'host' : 'localhost',\n 'port' : '6379',\n 'db' : 0\n }\n },\n 'job' : {\n 'default' : {\n 'redis' : 'default',\n 'jobs' : 'jobs', # redis list name for jobs\n 'failed' : 'failed', # redis list name for failed jobs\n }\n },\n },\n # ...\n}", "language": "python" } ] } [/block] # Initializing Job Extension To initialize job extension, type the following; [block:code] { "codes": [ { "code": "glim job:init\n# this will create a job.py on your app folder", "language": "shell" } ] } [/block] # Creating Jobs [block:code] { "codes": [ { "code": "# app/jobs.py\nfrom glim import Log\nfrom glim_extensions.job import Job\n\nclass HelloJob(Job):\n def run(self): # self.data is registered when a Job \n if 'author' in self.data.keys():\n Log.info('hello %s' % self.data['author'])\n else:\n Log.info('hello glim!')", "language": "python" } ] } [/block] # Producing Jobs [block:code] { "codes": [ { "code": "from glim_extensions.job import JobQueue\ndata = {\n 'author' : 'Aras Can Akin'\n}\nJobQueue.push(HelloJob(data)) # returns True or False\nJobQueue.push(HelloJob)", "language": "python" } ] } [/block] # Consuming Jobs [block:code] { "codes": [ { "code": "glim job:consume --name jobs\n# output\n# hello Aras Can Akin\n# hello", "language": "shell" } ] } [/block] # Failed Jobs In jobqueue extension, the failed jobs can be moved to an another redis list by raising exceptions. The `$ glim job:consume` command is looking for jobs to throw exception if a failed job occurs. [block:code] { "codes": [ { "code": "# app/jobs.py\nfrom glim import Log\nfrom glim_extensions.job import FailedJobError\nfrom glim_extensions.job import Job\n\nclass HelloJob(Job):\n def run(self):\n if 'author' in self.data.keys():\n Log.info('hello %s' % self.data['author'])\n else:\n raise FailedJobError()", "language": "python" } ] } [/block] In this example, when you push a job without author, the job will go to failed list in redis # Roadmap - the job system should work for many other message queue services like AWSQ, RabbitMQ, IronMQ, etc. - command for flushing job queue - add feature to push/pop scheduled jobs