Iterable and Iterator are objects which make it possible to traverse/loop through each item in containers like List, Dictionary, Tuple, Set etc. Most notably for loops, Comprehensions and Generator in python use Iterator and iterable. Throughout this post we will know how iterator and iteration works. How to create an Iterable and Iterator and performance gained by use of Iterator in program.

Are Iterable And Iterator Same Thing?

No, They are not. However, Iterable and Iterator are closely related. Iterables are those python object which implement special method __iter__() . On calling those special method, they return an Iterator object. If we put an Iterable in for loop, first it will get an Iterator — In which actually iteration happens. Lets see pseudo example of what happens under the for loop.

Then What Is An Iterator?

Iterators are basically python object which implements/have two special method in them. Also called as iterator protocol.

  • __iter__() : Return the iterator object itself.
  • __next__(): Return next item in container. If there are no further items, raise the StopIteration exception.

We know that we can loop over list using for-loop and get items in it. Then it must have same sort of iterator methods in it. Lets check it out with an example.

Looking inside the List object we can see that it implements that __iter__() function which makes it an Iterable but NOT YET ITERATOR. We can get Iterator by calling that special function.

There is another alias for __iter__() in python, iter() function. They both do same work but iter() function is little bit safe as it checks for returned value. This function raises TypeError if returned value is not an Iterator object. According to an Iterator protocol, Iterator object must implement __iter__() and __next__() special function and returns next item or StopIteration exception. Lets check it out.

As expected Iterator object has both methods.

Getting Items From Iterator.

Getting items from an Iterator is a fairly simple task. Just call __next__() method from an iterator until it raises StopIteration exception.

Finally, it raises an exception indicating all items have been traversed or iterated.

How For Loop Handles Exception?

To handle an exception, we catch an exception and do something in response of exception. But we don’t see any try-catch in for loop right?. To handle an exception ‘for-loop’ uses infinite while loop with try-catch under the hood to control the flow. Infinite loop is broken when exception occurs. Lets see that how its done.

In for-loop statement for item in example_list: we can also use example_list_iterator and get the same result. This is the reason that both Iterator object and Iterable implement __iter__() method.

Note: All Iterators are Iterable but Not all Iterables are Iterator directly. But with the help of iter() fucntion you can get Iterator out of any iterable.

So far we had some concept about Iterables and Iterators but what’s the benefit of using iterators in loop? and why not classical method of looping over list? Lets try out to find answers with example.

Suppose we are assigned a task to fibonacci series. In series, number that we need are less than 10.

1. Old School Method

First of all in this method we are going to create a list of Fibonacci numbers using function. After that we would compare each number for some condition and if number satisfies our condition, we print that number, else we end the process.

We know we are gonna need Fibonacci numbers less than 10 so series with length 10 is enough for us.

2. Using Pythonic way.

Now lets make an Iterator which gives Fibonacci number on every __next__() call.

Using Iterator to implement the same logic.

Lets compare the difference between two methods.

  • In old method, we first create all possible values in advance. Do some condition check and if condition fails break the loop.
  • In Iterator method we get one value at a time. Do some condition check and if condition fails break the loop.

One important thing to notice here in method with iterator is we don’t create value in advance. We create only when its needed. That Iterator is Lazy right? Its called Lazy Evaluation. Its called so because it remains idle all the time and gives only one value when called, then becomes idle again.

Where Is My Improvement?

Time and space complexity for this small problem is almost same for both method. But lets assume a scenario where we have to create object with size of 1MB instead of single integer and count in thousands rather than few. What would happen then????

1. With classical method

  • This single program would require several gigabytes of RAM to create and hold all the objects in advance.
  • If we don’t know the stopping condition in advance but during run-time, we are doomed.

2. With an Iterator

  • Only one object at a time would get created and after some condition check/operation this would go out of scope.
  • Previous object would get garbage collected and another new object will be created.
  • If condition fails and loop breaks, No more objects get created.
  • This iterator loop can go as much as needed with no penalty at all.

This can boost program performance by several factor for large object creation. In this way we can create and use Iterators in our program. But wait there is even simpler method than this to create iterator and they are called mighty “GENERATORS”. We will discuss about them in another post.

Bonus Point: Laziness Is The Key.