Sunil Chauhan

Copying objects in Python

Assignment statements in Python do not copy objects, they create bindings between a target and an object. This can make unintentional effect on mutable objects.
For example, below we are creating list ‘b’ with four inner lists using list ‘a’. See how changing element in list ‘a’ affect inner lists of list ‘b’.


>> a = [3, 4, 5] >> b = 4*[a] >> b >> [[3, 4, 5], [3, 4, 5], [3, 4, 5], [3, 4, 5]] >> a[ 0 ] = -7 >> a >> [-7, 4, 5] >> b >> [[-7, 4, 5], [-7, 4, 5], [-7, 4, 5], [-7, 4, 5]]

To avoid this, you have to create a copy of an object rather than a new reference. Two types of copy operations are applied to container objects such as lists and dictionaries:


a shallow copy and a deep copy.


A shallow copy creates a new object but populates it with references to the items contained in the original object.



>> a = [1,2,[3,4]]
>> b = list(a)
>> id(a) == id(b)
>> False
>> b.append(100)
>> b
>> [1, 2, [3, 4], 100]
>> b[ 2 ][0] = -100
>> b
>> [1, 2, [-100, 4], 100]
>> a
>> [1, 2, [-100, 4]]

Copy.deepcopy creates new object and recursively copies all the element ...

Python garbage collection

All objects are reference-counted. An object’s reference count is increased whenever it’s assigned to a new name or placed in a container such as a list, tuple, or dictionary, as shown here:


>> a = 37 >> b = a >> c = [] >> c.append(b)

This example creates a single object containing the value 37. ‘a’ is merely a name that refers to the newly created object. All other operations are simply creating new references to the object.


An object’s reference count is decreased by the del statement or whenever a reference goes out of scope (or is reassigned). e.g.



>> import sys
>> c = [1, 2, 3]
>> i = c
>> sys.getrefcount( c )
>> 3
>> del i
>> 2

The count returned is generally one higher than you might expect, because it includes the (temporary) reference as an argument to getrefcount().


Also, worth noting that for immutable data such as numbers and strings, the interpreter aggressively shares objects between different parts of the program in order to conserve memory. So, you will get reference count much higher than what you anticipate.

>> import sys >> a = 12 >> sys.getrefcount(a) >> 524 >> m = 10 >> sys.getrefcount(m) >> 795

When an object’s reference count reaches zero, it is ...

Mutable container as function default argument

Recently, I was debugging production code of my project and realised that code is not working as expected since at one place we have used mutable container as function default argument.


This has surprising effect on our code output. Let’s see what happen when we use list as default argument to function argument.


def foo(a=[]): a.ppend(5) return a

If you think whenever you call this innocent function, you will get a list with element 5. Then, you are wrong!!


>> foo() >> [5] >> foo() >> [5, 5] >> foo() >> [5, 5, 5]

To avoid this effect, use None as default value for argument 'a’.


This is happening because Python generates function compiled code object and default argument references at the time when you define the function, not when you are calling the function. In above example, as soon as we defined function ‘foo’, Python Interpreter generated code object and created one list object for identifier ‘a’. Every time we are calling function foo(), we are executing same code object and sharing same list object ‘a’ since it is now part of code object.

You can use ‘else’ clause with for loop

Python has peculiar syntax of using ‘else’ clause along with for loop. The “else” clause is executed if the loop completes normally, but not, if it was interrupted with a “break”.


for i in range(5): print i else: print “Hello World!!”

Output:


0 1 2 3 4 Hello World!!

Now, let’s try one more example with ' break’ keyword.


for i in range(5): print i if i == 4: break else: print “Hello World!!”

Output:


0 1 2 3 4

Personally, I like to avoid this contruct so my fellow developer should not get confused.

Closer look to Python Closures

Recently, I started taking Python sessions to some of my colleagues who are new to Python.

Below is some notes about how Closures are implemented in Python from that discussion.

First of all, what is Closure. I found the wikipedia definition of closures quite nice to understand fundamental concept. Here is first few lines from wikipedia:

In programming languages, a closure (also lexical closure or function closure) is a function or reference to a function together with a referencing environment—a table storing a reference to each of the non-local variables (also called free variables or upvalues) of that function. A closure—unlike a plain function pointer—enables a function to access those non-local variables even when invoked outside its immediate lexical scope.

Let's make things little bit simpler with the help of Python code.

def grand_pa(func): x = 20 def wrapper(): print "I am from parent value: %d" % x func() return wrapper

Here, grand_pa is a function which in turn takes function as its parameter. This function has one variable x with value 20 and one inner function wrapper.
Now, I will create one more function and pass this function to grand_pa as parameter.

def pa(): x = 10 ...

ack-grep tool: Blessing while working with CPython code

I just wanted to know the definition of PyType_Ready function inside CPython implementation and quite wondering where it is buried inside the code base.
Just found ack-grep package on my Ubuntu system and I must say I am impressed by it's blazing search speed. So, I think this tool is going to be my friend in coming days. Try once and you will not look back :-)

Push your code to Github

So, you are ready with your first program or project and want it to push on Github. There is great documentation for pushing code first time on Github website but this post describe the same from my eyes. There is no loss to clarify the same info to make Internet richer, Isn’t it :-)

Step 1: Goto create Repository page and make your own project repository on Github website. It will give option to create Readme file. It’s good practice to create readme file explaining your project and any requirement needed to run your program.

Step 2: Make a directory on your local system. Change your current directory to new directory.

Step 3:git init This will create hidden .git directory with necessary files.

Step 4:git clone https://github.com/USERNAME/Hello-World.git. You can find exact URL on your repository page.

Step 5:git remote add origin https://github.com/USERNAME/Hello-World.git This will create pointer origin that will point to our repository.

Step 6:cd .git Open config file. Changed url to url = ssh://git@github.com/USERNAME/REPO_NAME.git

Step 7: git add . Git has staging area. This command will add all modified files ...

Some missing links for Django app on Heroku

While playing with Heroku to upload django app, I found that there are some parts that are not clearly mentioned in document. I am putting here the missing links, so in future the lucky ones don’t have to struggle for these silly things:

1. Initialize your git at the root level of project (In the directory level where your procfile and requirements.txt files are situated).

2. Create your .gitignore file inside your django app (created through manage.py startapp command).

3. Your css and jquery files are not going to directly deployed to heroku. There is seperate settings we have to do for serving our css and javascript files from static folder. Blogs Waves and Maps and Matthewphiong gives a great details about how to create bucket on Amazon Web service S3 to store our static files and serving it. So I will not repeat the same.

4. Also for database backing service url, we first install django utility dj-database-url and in our settings.py file our entry will be for module dj_database_url. This might not be any tricky error at first sight but I made a mistake by entering dj-database-utility in my settings.py instead of dj_database_url.

Installing PostgreSQL on Ubuntu

I have just finished installing PostgreSQL on ubuntu, so these are the steps if someone else wants to do the same.

sudo apt-get install postgresql postgresql-client postgresql-contrib pgadmin3
Then to check if everythning installed correctly,type

psql --version


Since the only user who can connect to a fresh install is the postgres user, we have to set the username/password for this user
ALTER USER postgres WITH PASSWORD 'password';


You can use below command to reset the password
sudo su postgres -c passwd


For creating user other than postgres, you can follow create user on postgresql website and for creating read only user, you can explore this thread on stackoverflow

I still like you Google app engine but I have to go

Recently I have to stop using Google app engine for one of my hobby project. I feel really unfortunate for doing this since I am great fan of Google app engine. Let me tell you that Google app engine was something which provided me motivation to learn one of the language i.e. Python which is now my favourite. The ease of deployment, awesome dashboard to manage applications, free hosting till certain page views for ten applications with one gmail Id (Google people can increase this quota if requested) and its provided scalability was something which made me excited to try it as soon as I came to know about it. Even I hosted this blog on Google app engine using bloggart blogging system only because of that excitement.

But pity, after using it for nearly one and half year, I started getting feeling that this is not something with which I can continue developing some of my applications. There are certain problems I am facing while using it.

The first was use of comet for many functionalities in my web application. I wanted to implement facebook news ticker kind of functionality and user notification services for my application. Google ...