[Python] Problem with Python logging RotatingFileHandler in Django website

If seeing the log files are not rotated properly or correctly in Django web site, you most likely encounter the problem as the following article described:

Problem with Python logging RotatingFileHandler in Django website

"The log is done via RotatingFileHandler which is configured with 10 log files, 1000000 byte each. The log system works, but this are the log files I get:

-rw-r--r-- 1 apache      apache          83 Jul 23 13:30 hr.log
-rw-r--r-- 1 apache      apache      446276 Jul 23 13:03 hr.log.1
-rw-r--r-- 1 apache      apache      999910 Jul 23 06:00 hr.log.10
-rw-r--r-- 1 apache      apache         415 Jul 23 16:24 hr.log.2
-rw-r--r-- 1 apache      apache      479636 Jul 23 16:03 hr.log.3
-rw-r--r-- 1 apache      apache         710 Jul 23 15:30 hr.log.4
-rw-r--r-- 1 apache      apache      892179 Jul 23 15:03 hr.log.5
-rw-r--r-- 1 apache      apache         166 Jul 23 14:30 hr.log.6
-rw-r--r-- 1 apache      apache      890769 Jul 23 14:03 hr.log.7
-rw-r--r-- 1 apache      apache      999977 Jul 23 12:30 hr.log.8
-rw-r--r-- 1 apache      apache      999961 Jul 23 08:01 hr.log.9

As you can see it is a mess. Last log has been written to file hr.log.2 (Jul 23 16:24) instead of hr.log"

I did some survey and found the root cause is in here:
RotatingFileHandler bugs/errors and a general logging question
"The logging system is thread-safe but not safe
against multiple processes (separate Python instances) writing to the
same file. It certainly sounds like you need a scalable solution - and
having each script send the events to a network logging server seems a
good way of handling the scalability requirement. "

    These words above remind me a lot about the importance of synchronization when using multi-threading and multi-process. Also, scaling is another important item that a lot of people don't care. I want to highlight that we should be vigilant about these.

So, it is not your fault. Don't blame yourself. ( Sorry, I am kidding you ... )
Here is one solution for this issue. Please check out the following link ( it's written in Chinese ):
http://www.djangochina.cn/forum.php?mod=viewthread&tid=118752

[Django] The question of changing urlpattern dynamically

A couple of days my colleague gave me a quesiont about how to  change urlpattern dynamically in Django. Well, I indeed take some time to survey the way to do so even though we figure out alternatives to achieve the same result that we want. So, the following list is about the solution:


http://stackoverflow.com/questions/8771070/unable-dynamic-changing-urlpattern-when-changing-database
I perfer to adopt using middleware class to resolve this problem as follows:

An alternative method would be to either create a super pattern that calls a view, which in turn makes a DB call. Another approach is to handle this in a middleware class where you test for a 404 error, check if the pattern is likely to be one of your categories, and then do the DB look up there. I have done this in the past and it's not as bad as it sounds. Look at the django/contrib/flatpages code for a straightforward implementation of this approach.

For how to use middleware class, there is another link for the reference.
http://stackoverflow.com/questions/753909/django-middleware-urls

[Django] How to upload a file to via Django REST Framework?

Here is an example to use Parser: MultiPartParser to parse the media type: multipart/form-data. You also can take a look at this for more in detail: http://www.django-rest-framework.org/api-guide/parsers

Client Side via curl: ( Basically you can use either PUT or POST)
Command format:
curl -X PUT -H 'Content-Type:multipart/form-data' -F 'data=@/{your_file}' -u {account}:{password} http://{your server ip address}/

for instance:
curl -X PUT -H 'Content-Type:multipart/form-data' -F 'data=@/Users/administrator/Desktop/2015.12.01.yaml' -u danny:password http://192.168.1.100:8000/api/upload_func/file/



Server side ( Django REST Framework )

in your settings.py, add the following lines
==>
    'DEFAULT_PARSER_CLASSES': (
        'rest_framework.parsers.MultiPartParser',
    )


For your_api.py, you can refer to this example as below:

class UploadAction(APIView):
    permission_classes = (IsAuthenticated, IsAdminOrReadOnly)
    parser_classes = (MultiPartParser,)

    def put(self, request, format=None):
        try:
            # Read uploaded files
            my_file = request.FILES['data']
            filename = "/tmp/" + str(my_file)
            with open(filename, 'wb+') as temp_file:
                for chunk in my_file.chunks():
                    temp_file.write(chunk)
                temp_file.close()
        except Exception as e:
            return Response(status=status.HTTP_500_INTERNAL_SERVER_ERROR)
        return Response(status=status.HTTP_201_CREATED)


P.S: Please change 'data' to others if you put the different variable name in curl command!!

Reference:
http://stackoverflow.com/questions/21012538/retrieve-json-from-request-files-in-django-without-writing-to-file

If you want to get the content of uploaded file, you can directly use the. read() api.

Something like:

if request.FILES.has_key('data'):
    file = request.Files['data']
    data = file.read()
    #you have file contents in data




[Django] How to Find the Performance Bottlenecks in Your Django Views?

Do you want to find the performance bottlenecks in your Django Views? I recently found this article that introduces nice tools for us to check the performance issues in Django. How to do it? Please follow this URL:
http://djangotricks.blogspot.tw/2015/01/performance-bottlenecks-in-django-views.html

To setup line profiling, install line_profiler and django-devserver to you virtual environment:

(myproject_env)$ pip install line_profiler
(myproject_env)$ pip install django-devserver

Then make sure that you have the following settings in your settings.py or local_settings.py:

# settings.py
INSTALLED_APPS = (
    # ...
    'devserver',
)

MIDDLEWARE_CLASSES = (
    # ...
    'devserver.middleware.DevServerMiddleware',
)

DEVSERVER_MODULES = (
    'devserver.modules.sql.SQLRealTimeModule',
    'devserver.modules.sql.SQLSummaryModule',
    'devserver.modules.profile.ProfileSummaryModule',

    # Modules not enabled by default
    'devserver.modules.profile.LineProfilerModule',
)

DEVSERVER_AUTO_PROFILE = True  # profiles all views without the need of function decorator

I try these tools and get a bunch of analysis report from them, and I think they are useful and convenient. The following picture is one part of my result:

[Django] The summary of "Writing your first Django app" in offical Django document

Well, if we want to really get understanding something, the best way is to study and do by yourself. The official Django web site gives us a very good example for those who are the first time to use Django. So this article is just the quick summary of this "Writing your first Django app" as follows:

Writing your first Django app, part 1
Writing your first Django app, part 2
Writing your first Django app, part 3
Writing your first Django app, part 4
Writing your first Django app, part 5
Writing your first Django app, part 6
Advanced tutorial: How to write reusable apps

$ django-admin startproject r300
$ python -m django --version
$ python -c "import django; print(django.get_version())"
$ django-admin startproject R300
$ python manage.py startapp demo
$ python manage.py migrate
$ python manage.py runserver 0.0.0.0:8000

After finishing the above steps, we can get the file structure like the picture below:

This picture is about the polls application that looks like:

And this picture is about the admin application that is to manage polls data:

If you want to deploy this polls application to Apache web server with mod_wsgi module, please follow the previous article to install Apache and mod_wsgi. Then, we need to add the setting of /etc/apache2/site-available/default is as follows:

But, in this case, I have not resolved the permission for database and admin application. So, hope to fix these later.

P.S:
There is a very good django e-book as follows:
http://www.djangobook.com/en/2.0/index.html