December 9, 2010

Parsing tweets links, users and hash tags with django and python-twitter.

Hello again. Today I'm going to continue my previous post concerning django and twitter. Last time we've finished after synchronising with twitter. This time I'm gonna show you how to parse list of received tweets for rendering it in our template so that links,hash tags and users mentions are treated as clickable links. So here's the code, I've added some comments to clarify what we're doing currently:

def parse_tweets(tweets):
    import re
    
    result = []
    for tw in tweets:
        tweet = tw.text;
        #compile regexes as objects
        hash_regex = re.compile(r'#[0-9a-zA-Z+_]*',re.IGNORECASE) 
        user_regex = re.compile(r'@[0-9a-zA-Z+_]*',re.IGNORECASE)
        
        #first deal with links. Any http://... string change to a proper link
        tweet = re.sub('http://[^ ,]*', lambda t: "%s" % (t.group(0), t.group(0)), tweet)
        
        #for all elements matching our pattern...
        for usr in user_regex.finditer(tweet):
            
            #for each whole match replace '@' with ''
            url_tweet = usr.group(0).replace('@','')

            #in tweet's text replace text with proper link, now without '@'
            tweet = tweet.replace(usr.group(0),
                ''+usr.group(0)+'')

        #do the same for hash tags
        for hash in hash_regex.finditer(tweet):
            url_hash = hash.group(0).replace('#','%23')
            if len ( hash.group(0) ) > 2:
                tweet = tweet.replace(hash.group(0),
                    ''+hash.group(0)+''); 

        #append each tweet to our list. Here I'm creating a dict containing tweet's text and date
        posts.append({'tweet': tweet, 'date': s.pub_time})   
    return posts

And now to show it in template :

    {% for t in tweets %}

  • {% for k, v in t.items %}
    {% ifequal k 'summary' %}
    {{ v|safe }}
    {% else %}
    {% ifequal k 'date' %}
    {{v|timesince}} ago
    {% endifequal %}
    {% endifequal %}
    {% endfor %}
  • {% endfor %}
Probably there are better ways of doing this, but this one works for me. If you have any questions just write a comment or follow me via twitter

December 8, 2010

Check object's parameter value in django template

I'm just finishing probably the most complexed project I'll ever build in django, at least by myself. During the development I've encountered numerous problems, some of without satisfying solutions on the web. So now for the next few posts I will share the results of my research, thinking and developing. Topic for today is :

How to check value of object's parameters in a template ?

Why do I ever need to do this kind of thing you may think. Yeah that's what I thought till now. But just think of my exact example - We have a site with some objects, that people can vote on. If someone has already voted we show him the result - if not, we render a voting button. Things are easy when rendering static page that is refreshed after each vote. But what if we need (just as I did) it to be dynamic ? I'm pretty sure that this problem can be solved using custom templatetag. But I've thought about a simpler solution using filters. So let's start.
I have created a context processor storing IP address of user. In my template I'm returning list of objects in a for loop, and on each object I'm using my filter, with IP as parameter:

{{ submission|check_voting:IP_ADDRESS|safe }}

And now the filter part :

@register.filter
def check_voting(obj, ip):
    result = obj.check_vote(ip)
    if result:
        result = "
%s votes
" % obj.votes else: result = "" % (settings.SITE_URL, obj.get_absolute_url() ) return result

As you can see my filter takes two arguments - object on which it was called, and ip. Then I'm doing some processing on my object's method 'check_vote' that returns me boolean value depending on query result for vote by ip. According to it's value I'm either showing a facebook like button used for voting, or just plain vote counter.
Hope you'll find this useful.

November 8, 2010

Providing uniqueness of different objects across one database

Recently I needed to add facebook comments box for different pages (which were showing models of different classes). All of them needed some unique ID, to provide unique comments for each object. So at first I was thinking about applying GUUID for the db, but it looked like overcomplicating the problem. So I came up with this:

class GUUID(models.Model):
    guuid = models.PositiveSmallIntegerField(_(u"Dummy GUUID"), default=1)

It's a dummy model class with only one field (set to 1, but you can use it to provide some additional functionality). Ok, but where's our uniqueness ?

class Celebrity(BaseObject):
    uniq = models.IntegerField(blank=False, editable=False)

Here it is. Each model has a 'uniq' field set to integer field (u can use PositiveInteger if you want to). Then in save method of each object we add this :

    def save(self, *args, **kwargs):
        if not self.uniq:
            guuid = GUUID()
            guuid.save()
            self.uniq = guuid.id            
        super(Celebrity, self).save(*args, **kwargs)

Now when we save object, we check fo existence of 'uniq' field. If it is null, then we create a new GUUID object and set it's id as the value of 'uniq'.

October 18, 2010

Synchronizing django with twitter using django-syncr

Ok so you have your first django powered application running (let me guess - CMS ? Blog ? :) ) and now you'd like to make it more "web 2.0"-ish by adding your twitter feeds. Althought I personally think this app is useless and dumb people who pay me think differently. So let's start. First you'll need some python's dependencies.

python-oauth2
python-twitter and stay alert, cause you need the latest build (so don't use easy_install as you'll get the deprecated 0.6 version)
django-syncr

Then go to dev.twitter and (if you already haven't) register a new application. This will generate you "consumer_key" and "consumer_secret". Write them down. Now let's get back to our application.
Iinstall both python dependencies but django-syncr add as an application in settings.INSTALLED_APPS. The reason for this is that you need to edit a bit of code, since currently available syncr does not support OAuth recently introduced on Twitter. Next either find "get_access_token.py" in python-twitter package, or download this function from sources and send somewhere within the scope of python. Before sending it though, open and edit :

AUTHORIZATION_URL = 'https://api.twitter.com/oauth/authorize'
SIGNIN_URL        = 'https://api.twitter.com/oauth/authenticate'

consumer_key    = None
consumer_secret = None

if consumer_key is None or consumer_secret is None:

In place of consumer_key and consumer_secret insert proper (previously saved) values as strings. Save and close and execute it with :

"python get_access_token.py run"

You will get a unique link printed in the console, that you need to visit. Visit it and press "Enter". Now you will get two more values to store somewhere : "access_token_key" and "access_token_secret".

Okay, now open syncr.app.tweet and change `__init__` function to this :

def __init__(self, username, key, secret, token_key, token_secret):

        self.username = username
        self.api = twitter.Api(
                            consumer_key=key, 
                            consumer_secret=secret, 
                            access_token_key=token_key,
                            access_token_secret=token_secret)
        self.user_cache = dict()

Here you can have different approach of course. You can hardcode your key values in it, add them to settings or send them each time when synchronizing with Twitter. Decision is up to you. Okay, last improvements to django-syncr. Open syncr.twitter folder, and edit models.py file. Here change Tweet.tweeter_id field from PositiveIntegerField to either IntegerField(if you're running < 1.2) or BigIntegerField. We're finished with edits, so sync your database and add this code to the view/context_processor that will serve your tweets :

#1. create TwitterSyncr object
t = TwitterSyncr('twitter_username',
'your_consumer_key',
'your_consumer_secret',
'your_access_token_key',
'your_access_token_secret',)
#2. synchronize your TwitterSyncr object with existing Twitter user
t.syncUser('twitter_username')
#3. get tweets for the Twitter user (so it will import tweets from Twitter to your DB)
t.syncTwitterUserTweets('twitter_username')

#4. grab user from database and it's tweets
user = TwitterUser.objects.get(screen_name="twitter_username")
tweets = Tweet.objects.filter(user=user)

And voila ! You have a list of your most recent tweets available. Of course you can limit the amount of parsed tweets with [:number].

October 11, 2010

Recent months list in django

After a while I'm back with hot new stuff for you and me :) Recently I needed to select blog entries from 5 consecutive months (counting from current) to my footer archive list. The biggest problem was with the range of months in different years. After a lot of struggle I've came across a python module (probably you will have it in your default setup) `dateutil`, which gives new functionality to date related functions. Ok, so let's get to the code. My function was added to context processors to provide me functionality across whole site. Then we build a date object from current date. Next step is to run a generator expression (which I will speak about some more in the future as they're one of the most important features of python language together with coroutines and generator functions) for a increasing range of months (you can also use a list comprehension here if you need some additional lists functionality or a generator function. but I've tried to keep it simple). This gives us a generator object so we create a dictionary of it, with years as keys and corresponding months as dates. Because this dictionary provides us with numbers only, I've added additional function to translate them to string months values (note the brilliant Python's switch-case funcionality) here shown with Polish dictionary.

from datetime import datetime
from dateutil.relativedelta import relativedelta

def get_blog_archive(request):
    now = datetime.today()
    first_month = datetime(now.year, now.month, 1)
    previous_months = (first_month - relativedelta(months = months) for months in range(0, 5, 1))
    
    news_archive = {}
    for pm in previous_months:
        m = translate_month(pm.month)
        if news_archive.has_key(pm.year):
            news_archive[pm.year].append(m)
        else:
            news_archive[pm.year] = [m]
    m = translate_month(current_month)
    news_archive[current_year].append(m)
    
    return {'NEWS_ARCHIVE': news_archive,}

def translate_month(month):
    ret_month = ""
    months = {
            1: "Styczeń",
            2: "Luty",
            3: "Marzec",
            4: "Kwiecień",
            5: "Maj",
            6: "Czerwiec",
            7: "Lipiec",
            8: "Sierpień",
            9: "Wrzesień",
            10: "Październik",
            11: "Listopad",
            12: "Grudzień",
        }
    ret_month = months.get(month)
    return ret_month

Okay this code is nice but not really functional, beacuse we get months names as String (here with utf-8 chars) and its troublesome to parse them as urls. So here's a bit more functional approach :

    news_archive = []
    for pm in previous_months:
        d = datetime(pm.year,pm.month,1)
        news_archive.append(d)
    
    return {'NEWS_ARCHIVE': news_archive,}

And now in our template we just parse with for loop through list elements.
So that's all and stay tooned for some more advanced django examples. I'm doing a really big project right now but there's a ready shoutbox for django example, few implementations of django-registration, something about handling files and more :)

July 29, 2010

Conditional context processor for authenticated users

Recently I needed a context processor to display friends of my logged user. Userprofile is the name for extended user model. Here's the code :

def friends_list(request):
    try:
        user = request.user
    except User.DoesNotExist:
        user = AnonymousUser()
    
    userprofile = None
    try:
        userprofile = UserProfile.objects.get(user=request.user)
    except UserProfile.DoesNotExist:
        pass
    if userprofile:
        friends = Friend.objects.filter(friend_of=userprofile)
    else:
        friends = {}
    return {'friends': friends}

July 28, 2010

Python's lambda

Because I don't quite get what and when python's lambda function's can be of any use - I did some research. Then I've found this sleek way of calling one lined function with unnamed parameter :
>>> (lambda x: x*2)(3)
6
Pretty cool.

July 27, 2010

Scroll to top - the new way

Thing you will find useful at some point - how to scroll from random part of page to the top ? Here's a way how to obtain it with jQuery :

$(document).ready(function(){
  $('a[href=#]').click(function(){
    $.scrollTo(0,'slow');
    return false;
  });
});

But remember to include jQuery ScrollTo plugin in your imports. Now every link with hash as target will scroll your page back to the top with nice jquerish effect. Cooll huh ?

July 2, 2010

Random sort of array elements

Recently I came across problem of sorting randomly elements of some array. Here's what I came up with :
Pseudocode :

for each element in the array
calculate a random number with index range from 0 to array size -1
store the current element in a temporary variable
replace the current element in the array with the element at the random index
replace the element at random index with the element at temporary variable

Solution :
        int[] buttons = new int[16];

        for(int i=0; i<16; i++){
            buttons[i] = i;
        }

        int rand;
        int temp;
        Random random;

        random = new Random(System.currentTimeMillis());
        for (int i = 0; i < buttons.length; i++) {
            rand = (random.nextInt() & 0x7FFFFFFF) % buttons.length;
            temp = buttons[i];
            buttons[i] = buttons[rand];
            buttons[rand] = temp;
        }

June 11, 2010

Threads in Java

"Dining Philosophers" problem solved with Java and Semaphores.

import java.util.concurrent.Semaphore;
import java.util.Random;
import java.util.Vector;

public class Philosopher extends Thread
{
    private static final Random rand = new Random();
    private static int event=0;
    private static String binary="";
    private int id;
    private Semaphore sem;
    private static Vector vector = new Vector();

    public Philosopher(int i, Semaphore s)
    {
        id = i;
        sem = s;
        binary = binary + "0";
    }

    private void busy()
    {
        try
        {
            sleep(rand.nextInt(1000));
        } catch (InterruptedException e){}
    }

    private void thinking()
    {
        String str = "Philosopher " + id + " is thinking";
        vector.add( this.addToObject(System.currentTimeMillis(), event , str) );
        event++;
        busy();
    }

    private void eating()
    {
        String str ="Philosopher " + id + " is hungry and is trying to pick up his chopsticks";
        vector.add( this.addToObject(System.currentTimeMillis(), event , str) );
        event++;
        busy();
        str = "Philosopher " + id + " is eating";
        this.oneToBinary(id);
        vector.add( this.addToObject(System.currentTimeMillis(), event , str) );
        event++;
        busy();
        str = "Philosopher " + id + " finished eating, and puts away his chopsticks";
        this.zeroToBinary(id);
        vector.add( this.addToObject(System.currentTimeMillis(), event , str) );
        event++;
    }

    private Object[] addToObject(long t, int i,String s ){
        Object[] o = new Object[4];
        o[3] = s;
        o[2] = i;
        o[1] = binary;
        o[0] = t;
        return o;
    }

    private void oneToBinary(int i){
        binary = binary.substring(0,i) + "1" + binary.substring(i+1);
    }

    private void zeroToBinary(int i){
        binary = binary.substring(0,i) + "0" + binary.substring(i+1);
    }

    @Override
    public void run()
    {
        for (int i = 0; i < 10; ++i)
        {
            thinking();
            try
            {
                sem.acquire();
            } catch (InterruptedException e){}
            eating();
            sem.release();
        }
    }

    public static void main(String[] args)
    {
        final int N = 5;
        Semaphore sem = new Semaphore(N, true);
        Philosopher[] philosopher = new Philosopher[N];

        // Start the philosophers
        for (int i = 0; i < N; i++) {
          philosopher[i] = new Philosopher(i, sem);
          philosopher[i].start();
        }
        // Wait for them to finish
        for (int i = 0; i < N; i++) {
          try {
            philosopher[i].join();
          } catch(InterruptedException ex) {
            return;
          }
        }

        for (int i = 0; i < vector.size(); i++) {
            Object[] o = vector.get(i);
            System.out.printf("%d %d %s %s\n", o[0], o[2], o[1], o[3]);
        }
    }
}

June 8, 2010

Tip of the Day #2 - pointers

Remember
int* ptr;
*ptr = 10;
#but
ptr = new int;
#without ampersand and dereference !!

June 7, 2010

Constant pointers, pointers to constants ?

What is the difference between:
float fVar = 3.14;
const float* ptr1 = &fVar;
and:
float fVar = 3.14;
float* const ptr2 = &fVar;
? The first one is pointer to constant value, so we can't change the value of variable the pointer points to :
*ptr1 = 10;
The second one in contrary is a constant pointer to a value, and because of that we can't readdress it to point at different variable :
float fVar2;
*ptr2 = &fVar2;

June 3, 2010

Django Contrib Revealed - Flatpages

Very often while creating big portals there are static pages just wasting your time to be created. Django has a solution for this problem also - the Flatpages application inside Contribution package. This application allows you to manage such static pages from django-admin, and it let's you create templates using Django's template system. How to get started ?
- Add 'django.contrib.flatpages' to INSTALLED_APPS. Also you need to have django.contrib.sites activated since flatpages depend on this package.
- Add 'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware' to MIDDLEWARE_CLASSES.
This is how the flatpage model looks like :
from django.db import models
from django.contrib.sites.models import Site

class FlatPage(models.Model):
    url = models.CharField(max_length=100, db_index=True)
    title = models.CharField(max_length=200)
    content = models.TextField(blank=True)
    enable_comments = models.BooleanField()
    template_name = models.ChorField(max_length=70, blank=True)
    registration_required = models.BooleanField()
    sites = models.ManyToManyField(Site)
Okay so everything looks clear here I guess. For every flatpage we can specify either custom template, or use flatpages/default.html template (still creating the default template is also our responsibility !). Flatpage templates get a single context variable passed - flatpage, so we can use the Flatpage object's fields ie. {{ flatpage.title }}, {{ flatpage.content }}. Simple, but can spare lots of time :)

May 28, 2010

Django Contrib Revealed - Sites

Ok so the first element from django.contirbution I'm goin to write here about will be Sites package. What is Sites used for and what it's all about anyway ?
Let's say you have two sites connected together and sharing one database (i.e. community portal, and news site). You're using one login and authentication system there, one folder with uploaded pictures but different views for sites and different templates.
In settings file, community site has ID of 1 and news site's ID is equal to 2. How we would now build shared News model for both sites ?
from django.db import models
from django.contrib.sites.models import Site
from django.contrib.sites.managers import currentSiteManager

class News(models.Model):
    title = models.CharField(max_length=100)
    # ...
    site = models.ForeignKey(Site)
    objects = models.Manager()
    current_site = CurrentSiteManager()
As we can see from the code, we must add a field related to Site model in our class declaration (in this example it is many-to-one relationship). We can also notice two managers. The default objects is well known to us. It just serves .objects.all() . And for the second one it's a built in manager for serving objects specific to each site. So now News.objects.all() gives us all news (for example to feeds), and News.current_site.all() will return only News from current site. One thing to remember is that if our site-related field has different name than 'site' in CurrentSiteManager we need to specify this handler :
current_site = CurrentSiteManager('site_related_field')
And now for the views :
from django.conf.import settings

def simple_view(request):
    if settings.SITE_ID == 1:
        # code here
    else:
        # code here
But as you can see it's not the cleanest solution. It would be better if we had current site stored somewhere for the future use. Also, there is no need for refering to settings each time, because every Site model has a method .get_current() :
from django.contrib.sites.models import Site

def simple_view(request):
    current_site = Site.objects.get_current()
    #here we're using one of Site models field - domain.
    if current_site.domain == 'example.com':
        #code here
    else:
        # code here
Just to finish things up. Other field of Site model is name (which returns name of current site). And if you still prefere to use settings instead of get_current method, here's the code for that :
from django.conf import settings
from django.contrib.sites.models import Site

def simple_view(request):
    current_site = Site.objects.get(id=settings.SITE_ID)
    #here we're using one of Site models field - domain.
    if current_site.id == 1:
        #code here
    else:
        # code here

May 27, 2010

Parsing string

Yesterday I was checking my marks from C++ Programming subject (as it turned out, there's almost no chance I will pass it :/). Points from each lab are stored in spreadsheet, and when copied they looked like this :

"1 1 3 2 1 1 1 2 1 1 1 1 1 1 1 1,5 0,33 0,66 1 0,33 0,66 1 1 2 1 1 2 1 1 2 0,5 0,66 2 1 2 1 1 1 0 1"

Because exams are closing, I'm starting to get lazy. That's why instead of copying and summing all the values in google, I thought about using Python to parse through this string to get their sum. That's what I came up with :
#note using fsum to preserve float loss
fsum(float(n) for n in str.replace(',','.').split())
And here's second solution, since ',' is a legitimate decimal point for many people.
import locale
locale.setlocale(0,"po")
sum(map(locale.atof, s.split()))

May 26, 2010

Generic views part 1

Problem with start page of your web app is most common: "Do I need to create view for index.html, just to allow loading the page ?". Well you don't. Take a look at the code below :
from django.conf.urls.defaults import *
from django.views.generic.simple import direct_to_template

urlpatterns = patterns('',
      (r'^/$', direct_to_template, { 'template': 'index.html' })
)
What is this 'direct_to_template' import ? It's django's built-in view, that allows us to load template without specifying any view for it. More to come soon...

May 25, 2010

Tip of the Day #1 - in or not ?

I really love those small python's built in functions and operators. How to check if variable is in some collection ?

>>> z = x in s
>>> z
True
z will return True if x is in s. Isn't that cool or what ?

Faking URL values

Let's say we have a shop, with product urls looking like this:

http://myshop.com/motorbikes/choppers
http://myshop.com/bikes/bmx
http://myshop.com/skates/aggresive

But also we have a page for top selling products:

http://myshop.com/topsellers

, that uses the same view:
def products_list(request, parent_category, child_category):
How we can avoid writing another view for top sellers, that don't send either parent_ or child_ categories in URL's ?
urlpatterns = patterns('',
    (r'^topsellers/$', views.products_list, {'parent_category': 'topsellers', 'child_category': 'topsellers'}),
    (r'^(?P\w)/(?P\w)/$', views.products_list),
)

Also I've taken request to create application for simple management of orders, stocks, warranty operations from some hospital-stuff oriented company. So generally in about two months I need to learn how to write market-ready applications in python, with generating pdf's. I'm still not sure if that was a good idea (but you know how it is when pal asks you to do him a favour - especially if he is your former boss :P ).

May 22, 2010

Don't add unnecessary Context Processors

I guess we've all came across problem of using some attributes of our models in the templates. We know the solution for this : create your own context processor, add necessary values there, and voila ! We can use them across our site. But do we really always need to do this ?

Probably the most commonly used object is our User. We often need to check if the user is logged etc. But do we really have to add our user to a new Context Processor like this :
def user(request):
    if hasattr(request, 'user'):
        return {'user':request.user }
    return {}
?

Let's just take a look at our settings.py file. By default (in Django 1.2) TEMPLATE_CONTEXT_PROCESSORS variable looks like this :

TEMPLATE_CONTEXT_PROCESSORS = (
"django.contrib.auth.context_processors.auth",
"django.core.context_processors.debug",
"django.core.context_processors.i18n",
"django.core.context_processors.media",
"django.contrib.messages.context_processors.messages"
)

And looking into documentation we can find :

"user -- An auth.User instance representing the currently logged-in user (or an AnonymousUser instance, if the client isn't logged in)."

Voila again ! Orly ? Well not exactly. Yes, this gives us ability to check {{ user.username }} and check if a User is authenticated. But what if we have another Model defining our user, with additional fields (and inheriting from contrib User)? Don't panic. There's also solution concerning this fact. Let's say our extended user model is called UserProfile. Open settings.py once again and insert this line :

AUTH_PROFILE_MODULE = 'accounts.UserProfile',

where 'accounts' is the name of the app where you're storing your extended model. And now as Django Docs say :

"When a user profile model has been defined and specified in this manner, each User object will have a method -- get_profile() -- which returns the instance of the user profile model associated with that User."

May 20, 2010

Beginning C++

Two solutions for counting average of x < 100 numbers.
#1
int tab100];
    int n=0;
    int number=0;
    char input[10];

    while (n++ < 100)
    {     
       cout << "Give " << n << " number : ";
       memset(input, 0x00, 10);
       cin.getline(input, 10);
       number = atoi(input);
       if (number > 0)
           tab[n-1] = number;
       else
           break;
    }
    cout << average(tab, n-1) << endl; 
    getch(); 
    return 0;

#2
int main()
{
    const size_t COUNT = 100;
    int tab[COUNT];
    size_t n;   

    cin.tie(&cout);

    for (n = 0; n < COUNT; ++n ) {
        cout << "Give " << n+1 << " number : ";
        cin >> tab[n];

        if (!cin)
            break;
    }

    if (n > 0)
        cout << average(tab, n) << endl;
    
    getch();
    return 0;
}

Which one do you prefer ?

May 19, 2010

Python's all() function usage.

Recently I've came across superb python function all(). What it does is just selecting all items,variables,objects,etc fulfilling some requirements. Usage :

all(our_requirement)

Here's my code from Django-LFS (used on purple-cow.pl website). It's a function deactivating products if they have no stocks (also using my function get_stock_amount, which simply returns stocks of product). Still in development though, gotta add some control from management panel.

def deactivate(self):
        """If there are no stocks, deactivate the product. Used in last step of checkout.
        """

        inactive = False
        
        if self.is_variant():
            prod = self.parent
            inactive = all(var.get_stock_amount() == 0 for var in prod.variants.filter(active=True))
            if inactive:
                prod.active = 0
            prod.save()
        else:
            if self.get_stock_amount() == 0:
                self.active = 0
        
            self.save()

[Python] Joining two lists in a dictionary

Hello and welcome everybody.
First entry, so something pythonic for starters. Simple function for joining two lists in a dictionary. If anyone has better idea, leave a comment.



def mapper(keys, values):
n = len(keys)
return [dict(zip(keys, values[i:i + n]))
for i in range(0, len(values), n)]
stat4u