Javascript: JSON validator

JSON is fine and dandy when it works well, but damned if it's missing a comma or the bloody keywords are quoted in the wrong characters.

I had a problem with some (assumingly) well formed JSON, but it kept puzzled me for hours when a jQuery AJAX post kept silently failing.

Eventually I stumbled upon jsonlint.com, a nifty site that simply validates your JSON and formats it for easy reading.

My example JSON:

{
    'num_votes': 5,
    'up': 4,
    'down': -1
}

Can you spot what's wrong with it?

No? Neither could I... until I got fed up and changed the single quotes with double quotes for the hell of it.

AND IT BLOODY WORKED!

This is the now valid JSON:

{
    "num_votes": 5,
    "up": 4,
    "down": -1
}

FML.

8891
Oh well, back to it!

django: Require login on view

There's a real easy way of doing this in Django.

Sure you can manually write a check on every single view that requires a login to see if the user is an AnonymousUser:

if isinstance(request.user, AnonymousUser):
return HttpResponseRedirect(reverse('do-login'))

or you can just use a decorator and set the LOGIN_URL via the settings.

In your settings file, add:

LOGIN_URL = '/user/login/'

And then in your view:

from django.contrib.auth.decorators import login_required

@login_required
def some_protected_view(request):
# code goes here

That's it!

Note: HTTP header HTTP_X_FORWARDED_FOR will give you multiple addresses

During testing, this never showed up on the dev servers. The reason is that we only get one IP for HTTP_X_FORWARDED_FOR, unless we're behind a load balancer or HTTP proxy.

The correct method of using this is:

if 'HTTP_X_FORWARDED_FOR' in request.META:
question.ip = request.META.get('HTTP_X_FORWARDED_FOR').split(',')[0].strip()
else:
question.ip = request.META.get('REMOTE_ADDR', None)

This will get the client IP (first position) and strip out any spaces.

If there is no forwarded IP, it'll resort to using the REMOTE_ADDR header.

The code is written in Python but should be easily portable to any other language.

Sources

Python: Installing docutils on Windows

Django documentation pages require docutils to work. I thought this was gonna be another convoluted library build but it was ridiculously easy!

  • Download the latest snapshot of docutils here.
  • Extract the contents into any folder (just not within your Python setup)
  • Go to it in the console and then type:

python setup.py install

  • That'd be all!
  • Remove the docutils folder you just extracted since it's no longer needed.

SaM4M
BAM! Just like that and you're done!

[ Source ]

Python: Add MySQL support on Windows

I was starting work on migrating a project over to Django and needed access to an existing MySQL database. However, I couldn't as I was using Instant Django as a development server on Windows.

When trying to run Instant Django with MySQL, you're gonna see this error:

File "instant_django\Python27\lib\site-packages\django\db\backends\mysql\base.py", line 14, in <module>
raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: No module named _mysql

This is because Instant Django only ships with only sqlite support. If you want to run it with MySQL support, you'll have to add a few files your the setup.

Installation

Unfortunately, Instant Django is not installed so the installer for MySQL Python won't work because there is no registry value for Python.

  • Download and install the "MySQL-python-1.2.3.win32-py2.7.exe" from codegood.com.
  • Extract the contents of "MySQL-python-1.2.3.win32-py2.7.exe" (using WinRar or 7-zip) and you'll get a folder called "PLATLIB".
  • Move "_mysql.pyd" and "_mysql_exceptions.py" to "instant_django\Python27\Lib\site-packages\PIL".
  • Move "MySQLdb" to "instant_django\Python27\Lib\site-packages".
  • Try to start your Django server and it should actually work.

Just in case

Instant Django is running the 32bit version of Python 2.7 (at time of writing), which cannot load the MySQL-Python x64 libraries.

If you're trying to install the 64-bit version, you'll get this error:

File "instant_django\Python27\lib\site-packages\django\db\backends\mysql\base.py", line 14, in <module>
raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: DLL load failed: %1 is not a valid Win32 application.

Now that you're over this hurdle, happy coding!

1299240990_4-vault-jump

[ Download ]

Java: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

If you've ever encountered this error, it means your project (or the application you're trying to run) has used too much memory.

To remedy this, find the script that is executing it and add in a new argument "-Xmx1024m".

This will set the maximum heap size to 1024mb. You can use any other value, but I was parsing a big Android method trace file using traceview.

For example, "traceview.bat" contains this line at the end:

call %java_exe% -Djava.ext.dirs=%javaextdirs% -Dcom.android.traceview.toolsdir= -jar %jarpath% %*

Simply add in the argument:

call %java_exe% -Xmx1024m -Djava.ext.dirs=%javaextdirs% -Dcom.android.traceview.toolsdir= -jar %jarpath% %*

And it'll work properly.

X6PLT 
Java, "it just works".

Android: How to use scaled pixel or font point sizes as pixel dimensions for Paint.setTextSize() and Canvas.drawText()

Android supports a whole heap of dimensions such as density independent pixels (dp), scaled independent pixels (sp), font points (pt), raw pixels (px), millimetres (mm) or inches (in). See more information about them in the documentation.

However, rather than having to calculate all of those manually and causing a whole lot of mindfuck to the dev community, they've provided some rather simple methods of working out these values.

sp91d
Google have prevented this backlash from developers.

Using Resources.getDimensionPixelSize(), we can convert any dimension into a raw pixel value. This ensures that Paint.setTextSize() will display the same across all devices.

Creating the resource

First, create an Android XML file resource under  "res/values" called "dimensions.xml" of resource type "value".

Using the editor, add a new Dimension of any name and the value of your choice (ie. 20pt).

<?xml version="1.0" encoding="utf-8"?>
<resources>
<dimen name="custom_text_size">16sp</dimen>
</resources>

Using the dimension

Now to use this value in your code.

m_paint = new Paint();
m_paint.setTextSize(yourActivity.getApplicationContext().getResources().getDimensionPixelSize(R.dimen.custom_text_size));

Then just use that Paint object in the Canvas.drawText() call.

Remember to use Paint.ANTI_ALIAS_FLAG to ensure that the text looks good!

[ Android dimensions docs, StackOverflow ]

Android: Control media volume instead of ringer volume with volume buttons

It makes sense that Android, by default, leaves the volume controls to change the ringer volume. Not all apps play sounds or videos so it'll be annoying to press home every time you want to change the volume.
However, if your app plays sounds at irregular intervals then you'd notice that the volume can only be controlled while they are playing.
So how can you change the volume of the sound effects while you're in the app? Amazingly, it involves no overriding of event listeners for buttons or implementing of listeners.
During your Activity.onCreate() call, simply add this magical one liner:
this.setVolumeControlStream(AudioManager.STREAM_MUSIC);
That's it! Now the volume buttons will behave.
I could not find that in the documentation for the life of me, but it was scattered across a few StackOverflow questions.
kim jong beiber
This magical one liner will leave you feeling awesome, just like Kim Jong Il Beiber.
[ Source ]

Microsoft Office: How to print background of Word document to PDF

PDF's are great when they work, but one of the small issues with printing in general is that the process tends to strip out some stuff you want as well.

Background colours or images may be lost upon printing, so to keep them, go to File > Print > Options and tick the "Background colours and images" option.

image

image

That should generate a beautiful PDF with background colour for you.

django: Modify the model admin form for a specific model

The automatically generated django admin forms are already quite customisable already, but sometimes you need to add a new element or piece of information that just isn't part of the form or model.

That's where its easier to override the template a little.

Place the file in a template path in your project or app, in the following directory structure and filename:

"admin/your_app_label/your_model_name/change_form.html"

The model name should be in lower case.

Now in the template file:

{% extends 'admin/change_form.html' %}

{% block after_related_objects %}
{{ block.super }}

Oh harro!
{% endblock after_related_objects %}

In your django installation folder, see "contrib\admin\templates\admin\change_form.html" for what block names you can override.

Another method would be to specify the template file in the model.

class MyModelAdmin(admin.ModelAdmin):
# A template for changing the model admin form
change_form_template = "myapp/blah_change_form.html"

Just remember, do not override too much stuff. Stick to modifying small blocks and always remember to display {{ block.super }} in case anything changes in future revisions.

peniscolada12
A simple example of what happens when you change too much.

[ Source ]

python: How to read CSV files

A common task for projects is to read in data from external sources. One common source is a CSV spreadsheet file, where data is separated by commas.

Python provides a library called "csv" and it seems to do the job pretty well.

import csv

try:
file = open("filename.csv", 'r')
except IOError:
print "There was an error opening the file"
return

first_run = True
reader = csv.reader(file)

for row in reader:
if first_run:
# This row is the header row
first_run = False
continue

print row

That's all there really is to it. The rest of the code really depends on how you want to treat the data. Be good to it.

qn2nw1

To write to the CSV file, see source.

[ Source ]

django: Mark field as required on a form

A simple little feature which is lacking from the automatic form generation is marking something as required.

It's a bit of a shame that you have to override the whole form to get this working, but it gives more flexibility to how you want the required markings to be displayed.

{% for field in form %}
<div>
<label for="{{ field.auto_id }}">
{% if field.field.required %}<span class="required">{{ field.label }}</span>
{% else %}{{ field.label }}{% endif %}
</label>
{{ field }}
{{ field.errors }}
</div>
{% endfor %}

[ Source ]

django: Use a text area in your form

It was just one of those days when I stumbled upon this little gotcha.

I was a bit baffled when my textarea field would not show up on the form.

class YourForm(forms.Form):
name = forms.CharField(maxlength = 20)

# Textarea done WRONG
#address = forms.Textarea()

# Textarea done right!
address = forms.CharField(widget = forms.widgets.Textarea())

A simple mistake and a little tricky to figure out because it fails silently.

Sparta
THIS IS SPARTA DJANGO!

[ Source ]

django: Caching RSS Feed objects

Django's syndication library works pretty well, it's easy to extend and use, but caching is another story.

Because you don't have access to the view, you can't use fragmented template caching.

Because the Feed class is not a view, we're a bit stuck on that too.

Luckily the class is "callable", meaning we can catch it as it's called.

class YourFeed(Feed):
def __call__(self, request, *args, **kwargs):
# Cache the RSS feed
# It has to be done this way because the Feed class is a callable, not a view.
cache_key = self.get_cache_key(*args, **kwargs)
response = cache.get(cache_key)

if response is None:
response = super(YourFeed, self).__call__(request, *args, **kwargs)
cache.set(cache_key, response, 900) # cache for 15 minutes

return response

def get_cache_key(self, *args, **kwargs):
# Override this in subclasses for more caching control
return "%s-%s" % (self.__class__.__module__, '/'.join([ "%s,%s" % (key, val) for key,val in kwargs.items() ]))

This method should work for subclasses of YourFeed also.

tumblr_lhccqjy43R1qgewq6o1_500
Nothing left to do but DANCE!

[ Source ]

Android: Performance profiling and how to simple ways to improve speeds

I was wondering why my game, SlowPoke Blitz, was severely underperforming when the scoreboard appeared. An easy way to figure out what processes were eating up CPU time was to view the trace log.

Note: Code benchmarking is a different process where you compare the performance of applications. Profiling helps you determine bottlenecks and inefficient coding.

To begin profiling your code, you'll need to ensure you have met these requirements:

  • Permission to write a trace log file
  • An emulator which has internal memory specified
  • Decided a suitable starting point for logging
  • Decided a suitable ending point for logging

Generating Trace Log

To begin profiling your code, you'll first need to generate the trace file.

In your code, add in:

Debug.startMethodTracing("traceFile");

I place this in Activity.onCreate(), but you can do this anywhere you think is suitable.

Next, find a fitting end point for the code you're profiling. Then paste in:

Debug.stopMethodTracing();

Now you'll need to open up "AndroidManifest.xml" and add:

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

Run your application as normal and then exit. It'll run a little slower than usual, but that's OK because the results we're after are given in percentages.

Retrieving the trace file

Using the Android SDK, you should have access to "adb" and "traceview".

Using ADB, you can extract the file from the emulator using:

adb pull /sdcard/traceFile.trace C:\trace\

This will download and save the file to C:\trace\traceFile.trace

Using traceview.bat, type:

traceview C:\trace\traceFile.trace

Note: you need to specify the full pathname.

This will start the Traceview program.

Reading the trace data

image
Jibberish? Certainly not!

The bars at the top help you visualise the CPU time which each thread consumes.

The tree list at the bottom indicates which functions are called, ordered by the amount of processing time it takes.

You can see that GameCanvasView:DrawThread.run() is eating a whopping 70% of the CPU time. That's ok, because it's a game which redraws often and encompasses the 2 chunky functions in spot #2 and #3, GameCanvasView.onDraw() and GameCanvasView.drawScoreboard().

Wait, drawScoreboard() is #3? Why? It's only being shown for about 4 seconds at the end! Why is it using up so much CPU?

Wow, ScoreboardData.draw() is #4, using 38.2% of the CPU draw time. Something just isn't right.

Expading the ScoreboardData.draw() tree shows some interesting information. It shows that String.format() is proving to be VERY expensive in the display process.

That explains why the rest of the Traceview screenshot showed mainly java.lang.String and java.util.Formatter functions.

Speeding up the app

Be very careful with string allocations!

The code before:

public long draw(Canvas c) {
c.drawText(String.format("2x %d = %d", comboHit2, comboHit2 * 2), x, y, m_paint);
c.drawText(String.format("3x %d = %d", comboHit3, comboHit3 * 3), x, y, m_paint);
}

I found a very easy way to reduce the CPU usage. Rather than rebuild the String every 25ms (40 times a second), I found Strings which were unlikely to change and moved them out of the draw() method.

The code after:

private String str2HitCombos;
private String str3HitCombos;

public void gameFinished() {
str2HitCombos = String.format("2x %d = %d", comboHit2, comboHit2 * 2);
str3HitCombos = String.format("3x %d = %d", comboHit3, comboHit3 * 3);
}

public long draw(Canvas c) {
c.drawText(comboHit2, x, y, m_paint);
c.drawText(comboHit3, x, y, m_paint);
}

Once the game ends, I built the Strings in gameFinished() and only read them in the display. This dramatically improved the rendering speed of the scoreboard!

image
This was the end result. I forgot to take an intermediate screenshot.

After performing another trace, the call to drawScoreboard() dropped off the first page of the Traceview to below 4.8%, along with all the java.lang.String and java.util.Formatter methods.

Such a simple change and the results above show it's well worth it! I also took the time to make a few other changes around the place with similar design patterns.

Only initialise when you need to

Another method called by onDraw(), drawHud(), was eating up 17% CPU. This method simply displays the bar indicating how much time was left.

In that method, I was initialising Rect objects over and over and giving them the same values. Those values could have been determined when the View was created, so I moved them out of the draw loop and set them once.

After the simple change, it dropped to 5.0% CPU usage.

Avoid internal use of get/set methods

This is straight off the Android documentation. They're expensive and you'd rather be using the variables directly rather than through a get/set method.

I've personally extended this tip a bit more. Although frowned upon in the encapsulation world, I've declared more class variables to be public than I should rather than forcing external classes to go through getter and setter methods.

Remove defensive code once your app is grown up

Initially, my BitmapCache was a bit experimental. I wrote defensive code in case the test cases failed so I had some sort of fallback in debugging.

private HashMap<Integer, Bitmap> m_cache = new HashMap<Integer, Bitmap>();

public static Bitmap loadBitmap(int resid) {
if (m_cache.containsKey(resid)) {
return m_cache.get(resid);
}

Bitmap bmp;
bmp = BitmapFactory.decodeResource(activity.getApplicationContext().getResources(), resid);
m_cache.put(resid, bmp);
return bmp;
}

Now this may seem fine, but the defensive call to HashMap.containsKey() becomes VERY expensive if you're calling this every 25ms.

Especially when all the images are pre-loaded when the game starts, this becomes unnecessary.

Rewriting it to be less defensive, it now looks like:

public static Bitmap loadBitmap(int resid) {
Bitmap bmp = m_cache.get(resid);

if (bmp == null) {
bmp = BitmapFactory.decodeResource(activity.getApplicationContext().getResources(), resid);
m_cache.put(resid, bmp);
}

return bmp;
}

Now with the assumption that bitmaps are loaded, the code is much more efficient.

Cast primitives correctly

Assuming elapsedTime is an int, this:

String.valueOf((double) elapsedTime / 1000);

is slower than:

String.valueOf(elapsedTime / 1000.0);

Try/catch is expensive

Another silly mistake I made was using try/catch around something which didn't always need it.

try {
if (sleep_time > 0) {
Thread.sleep(sleep_time);
}
}
catch (InterruptedException e) {
}

This would stupidly create an expensive try/catch block around the if case even though (potentially) the Thread.sleep() is not called.

After:

if (sleep_time > 0) {
try { Thread.sleep(sleep_time); }
catch (InterruptedException e) { }
}

Summary

Of course, there are many more ways to increase the speed of your code. These are the only silly mistakes I could remember to share.

I tried not to cover topics which are already in the documentation page for optimising performance. You should also apply those methods to your application.

Sources

Android: Using postInvalidate() to trigger onDraw() with SurfaceView

When trying a different approach to controlling the onDraw() timing in my app, I stumbled upon a little roadblock which was not mentioned in the documentation.

I tried to use postInvalidateDelayed() to control the amount of time it took for the next redraw to happen, but for some reason it seemed to be ignored. I've even tried all invalidate(), postInvalidate() and postInvalidateDelayed() with threading.

invalidate() had to be called in the UI thread, however postInvalidate() and postInvalidateDelayed() could be called from any thread.

The reason is the SurfaceView class calls the setWillNotDraw() method to prevent the normal drawing methods of the View class.

That method has to be called in the UI thread, however it cannot be called in the constructor as the Surface has not yet been created.

You should call setWillNotDraw(false) during the surfaceCreated() stage of initialisation. Doing that, you sacrifice a bit of performance that the SurfaceView offers.

If you'd rather wait for a specific time to disable the SurfaceView custom drawing, use an AsyncTask.

new AsyncTask<Void, Void, Void>() {
@Override
protected Void doInBackground(Void... params) {
return null;
}

@Override
protected void onPostExecute(Void result) {
// This allows View.postInvalidate() to work with a SurfaceView
setWillNotDraw(false);
// This is not needed as setWillNotDraw() forces a redraw.
// m_drawScoreboard.postInvalidateEnabled = true;
}
}.execute();

That said, I still found it was better (performance-wise) to rewrite my code so it supported timed drawings in the drawing thread without changing "will not draw".

[ SurfaceView ignoring postInvalidate()? and Extended SurfaceView's onDraw() method never called ]

 
Copyright © Twig's Tech Tips
Theme by BloggerThemes & TopWPThemes Sponsored by iBlogtoBlog