Thursday, March 15, 2018

Drawing sound (as a waveform in pygame).

I recently gave an example of pygame sound generation examples, and a few people asked for more. So... here we go!

There's an infinite number of ways to visualize sound. A classic way is to draw it as a waveform.
Sound samples drawn as a Waveform. Scaled into a 320x200 sized Surface.

A sound could be made up of 44100 samples per second. Where each sample is often a 16 bit number (or 8bit or a 32bit floating point).
Python comes with a built in array for efficiently storing numbers. We can store samples in there, with integers between -32768 and 32768. This is a signed 16bit(2 byte) number. Two to the power of 16 is 65536, and if we divide that by two we see the minimum and maximum value that can hold. pow(2, 16) / 2 == 32768.

Below is the annotated code, also with an example of an array of samples representing a square wave.
We loop over the samples and draw a line from the previous sample to the current one. 0 to 1, 1 to 2, 2 to 3, ... N-1 to N.

You can also find it in the pygame CookBook at
(maybe easier to read than on this blog)

# python built in array type is useful for storing sound data.
import array
# a square wave tone, with sample values between -32766 and 32766.
samples = array.array('h', [32766, 32766, 32766, 32766, 32766, 32766,
32766, 32766, 32766, 32766, 32766, 32766, 32766, 32766, 32766,
32766, 32766, 32766, 32766, 32766, 32766, 32766, 32766, 32766,
32766, 32766, 32766, 32766, -32766, -32766, -32766, -32766,
-32766, -32766, -32766, -32766, -32766, -32766, -32766, -32766,
-32766, -32766, -32766, -32766, -32766, -32766, -32766, -32766,
-32766, -32766, -32766, -32766, -32766, -32766, -32766])

def scale_samples_to_surf(width, height, samples):
    """ Returns a generator containing (x, y) to draw a waveform.

    :param width: width of surface to scale points to.
    :param height: height of surface to scale points to.
    :param samples: an array of signed 1 byte or signed 2 byte.
    assert samples.typecode in ['h', 'b']
    # precalculate a bunch of variables, so not done in the loop.
    len_samples = len(samples)
    width_per_sample = width / len_samples
    height_1 = height - 1

    if samples.typecode == 'h':
        # for array typecode 'h', -32768 to 32768
        factor = 1.0 / 65532
        normalize_modifier = int(65532 / 2)
    elif samples.typecode == 'b':
        # for array typecode 'b', -127 to 127
        factor = 1.0 / 256
        normalize_modifier = int(256 / 2)

    return ((
        int((sample_number + 1) * width_per_sample),
            (1.0 -
                (factor *
                    (samples[sample_number] + normalize_modifier)))
            * (height_1)
    for sample_number in range(len_samples))

def draw_wave(surf,
              wave_color = (0, 0, 0),
              background_color = pg.Color('white')):
    """Draw array of sound samples as waveform into the 'surf'.

    :param surf: Surface we want to draw the wave form onto.
    :param samples: an array of signed 1 byte or signed 2 byte.
    :param wave_color: color to draw the wave form.
    :param background_color: to fill the 'surf' with.
    assert samples.typecode in ['h', 'b']
    if background_color is not None:
    width, height = surf.get_size()
    points = tuple(scale_samples_to_surf(width, height, samples))
    pg.draw.lines(surf, wave_color, False, points)

# Here we should how to draw it onto a screen.
waveform = pg.Surface((320, 200)).convert_alpha()
draw_wave(waveform, samples)
screen.fill((255, 255, 255))
screen.blit(waveform, (160, 100))

Wednesday, March 14, 2018

Sound generation pygame examples.

Here's a few sound generation examples with pygame (and no numpy/scipy).

If there's interest I'll expand this into a bigger example? Let me know.

All the basics for making a music program (sampler/synth).
  • some sample rate conversion,
  • bit rate conversion
  • tone generation using generators (square wave)
  • python arrays used as buffers for pygame.Sound (zero copy).

""" Some examples for generating and converting sounds for pygame.

Python 2.7, 3.6

    - a simple 'square wave' generated
    - resampling sample rates (eg, 8363 to 44100)
    - using built in python array for making pygame.Sound samples.
    - samples at different bit sizes
    - converting from signed 8 to signed 16bit
    - how initializing the mixer changes what samples Sound needs.
    - Using the python stdlib audioop.ratecv for sample rate conversion.

Square Wave
MOD (file format)


array (python stdlib)
wave (python stdlib)
audioop.ratecv (python stdlib)


from array import array
import pygame as pg

class Tone(pg.mixer.Sound):
    """This generates a 'Square wave' with a generator.

    Then creates an array of samples, and passes that into pygame.Sound.

    def __init__(self, frequency, array_type, volume=.1):
        self.frequency = frequency
        if array_type == 'b':
            # we have to convert the 1 byte 'b' samples to 2 byte 'h'.
            samples = self.signed_char_to_signed_short(
        elif array_type == 'h':
            samples = self.make_samples_h()
            raise ValueError('array_type not supported')

        pg.mixer.Sound.__init__(self, buffer=samples)

    def make_samples_b(self):
        """ Builds samples array between -127 and 127.
            Array type 'h'.
        mixer_frequency = pg.mixer.get_init()[0]
        mixer_format = pg.mixer.get_init()[1]
        period = int(round(mixer_frequency / self.frequency))
        max_amplitude = 2 ** (abs(mixer_format) - 1) - 1
        max_amplitude = int(max_amplitude / 256)
        # print(f'mixer_frequency:{mixer_frequency}, mixer_format:{mixer_format}')
        # print(f'period:{period}, max_amplitude:{max_amplitude}')

        # 'b' array is signed char, 1 byte
        samples = array('b',
            (max_amplitude if time < period / 2 else -max_amplitude
                for time in range(period))
        return samples

    def signed_char_to_signed_short(self, b_samples):
        """ Converts 1 byte signed char samples to 2 byte signed short.

            127 to 32767
        # just a simple linear conversion.
        factor = int(32767 / 127)
        return array('h', (sample * factor for sample in b_samples))

    def make_samples_h(self):
        """ Builds samples array between -32767 snd 32767.
            Array type 'h'.
        mixer_frequency = pg.mixer.get_init()[0]
        mixer_format = pg.mixer.get_init()[1]
        period = int(round(mixer_frequency / self.frequency))
        max_amplitude = 2 ** (abs(mixer_format) - 1) - 1
        # print(f'mixer_frequency:{mixer_frequency}, mixer_format:{mixer_format}')
        # print(f'period:{period}, max_amplitude:{max_amplitude}')

        # 'h' array is signed short, 2 bytes
        samples = array('h',
            (max_amplitude if time < period / 2 else -max_amplitude
                for time in range(period))
        return samples

class Sample(pg.mixer.Sound):
    """ For playing a sample.

    Takes a file, and reads it in as 8bit signed data.

    Then converts it to the 16bit signed size the pygame.mixer needs.
    def __init__(self, fname, volume=.1):
        with open(fname, 'rb') as f:
            samples = self.signed_char_to_signed_short (
            pg.mixer.Sound.__init__(self, buffer=samples)

    def signed_char_to_signed_short(self, b_samples):
        """ Converts 1 byte signed char samples to 2 byte signed short.
            127 to 32767
        # just a simple linear conversion.
        import time
        factor = int(32767 / 127)
        samples = array('h', (
            max(sample, -127) * factor if sample < 0 else
            min(sample, 127) * factor
            for sample in b_samples))
        return samples

def fetch_example_mod_file(mod_fname):
    """ Grab a file that has a sound samples in it from the net.

    'MOD is a computer file format used primarily to represent music,
    and was the first module file format. MOD files use the ".MOD"
    file extension, except on the Amiga which doesn't rely on
    filename extensions, instead it reads a file's header to
    determine filetype. A MOD file contains a set of instruments in
    the form of samples, a number of patterns indicating how and when
    the samples are to be played, and a list of what
    patterns to play in what order.'
    import os
    url = ''

    if not os.path.exists(mod_fname):
        import urllib2
        print ('Fetching %s .mod into file: %s' % (url, mod_fname))
        data = urllib2.urlopen(url).read()
        with open(mod_fname, 'w') as modf:

def resample(mod_fname):
    """ An example of resampling audio to a different framerate.

    eg, from 8363 one byte samples per second to
        44100 two byte samples per second.
    import audioop
    import wave
    from io import BytesIO

    in_framerate = 8363
    in_sampwidth = 1
    in_nchannels = 1

    out_framerate = 44100

    num_seconds = 5
    with open(mod_fname, 'rb') as f:
        # Throw away the start data of this mod file.
        #   Better samples later on.*2)
        in_frame_data = * num_seconds)

    newfragment, newstate = audioop.ratecv(

    # print(f'len(newfragment):{len(newfragment)}')
    # A perfect conversion is not possible, because the sample
    #   rates do not divide equally. However, the number
    #   of samples should be close.
    assert (out_framerate * num_seconds) - len(newfragment) < 10

# Converting between modo and stereo?
#   audioop.tomono and audioop.tostereo

# How to draw a wave form?
#   using pygame.draw.lines transforming audio into
#   Surface space.
#   Meaning, scaling audio samples into a particular
#   sized part of the screen.

# More sound generator types.
#   Saw tooth.

if __name__ == "__main__":
    pg.mixer.pre_init(44100, -16, 1, 1024)

    pg.display.set_caption('Playing square wave, 808 frequency')
    pg.display.set_mode((320, 200))

    mod_fname = 'outrun_3.mod'


    # play on repeat, -1 means loop indefinitely.
    if 0:
        Tone(frequency=808, array_type='b').play(-1)

    if 0:
        except IOError:
            print ('no %s' % mod_fname)
    if 0:

    if 1:

    going = True
    while going:
        for e in pg.event.get():
            if e.type in [pg.QUIT, pg.KEYDOWN]:
                going = False

Tuesday, March 13, 2018

pypy sprint

Been doing a bit of reading this morning on pypy things.

Trying to prepare myself a bit for the coming 'winter sprint'. One topic I'm interested in for the sprint is finishing off remaining issues for pygame running on pypy. Probably not the most common reason for people to go to the Swiss mountains I guess! So this means learning about the cpyext part of pypy... the one which allows it to use the python C API, and modules that use it.

Open source sprints are great. A bunch of people get together trying to make things. A very rare time when people get to meet other people-from-the-internet. It's a great place to learn, and to see how projects you're interested in are made.

I guess I'm a little bit too excited, and started playing with pypy early. Below are some notes from my investigations finding out how to do things.

Getting started, finding docs.

There's a getting started guide for pypy which has some basic notes on running tests, and compiling stuff.

Also, cython has a pypy section in their user guide -

cpyext is the module which implements the python C API in pypy.

A wiki page called 'c-api' contains some documentation including a talk on cpyext. It's a little bit dated (from 7 years ago), but still interesting reading(and watching).

This wiki page was interesting:  "In which we document cpyext compatibility progress, this time with cython and pandas."

It's a blog of development of pypy cpyext getting numpy and pandas features working. From this I noticed a few interesting techniques... like running tests separated processes. Very useful when debugging. There is also an older document called 'Adventures in cpyext compatibility (ended)'. With even more notes on issues that came up, along with their work arounds.

Of course the Python C API docs are also useful when one wants to do Python C API things.

Use the source (Luke).

But the best source of information on cpyext seems to be the source code itself.

Lots and lots and lots of tests. And lots and lots of code. Bed time reading perhaps.

It seems to be a whole implementation of CPython, with bridges into pypy. But with tests. Lots and lots of tests for the python C API.

What are useful debugging techniques?

CPython has a thing called a debug build which is useful.

I don't know of any special debug tools for pypy (gdb extensions or such like).

Wheels on pypy?

I found this page "pypy binary wheels for some popular packages", and a blog post about it.

It seems pypy doesn't compile on the ancient centos 5 that many linux uses. Perhaps the new manylinux which uses a newer linux will work.

It seems the piwheels project (compiles wheels for raspberry pi automatically) doesn't have pypy support yet. piwheels is really great, in that it makes lots of ARM builds available so people don't need to compile stuff themselves on the raspberrypi.

No idea if windows, or mac binary wheels work for pypy. Without binary wheels for these platforms, most pypy users won't be able to install a C based library like pygame on pypy.

Which platform?

It seems linux amd64 is the best supported, but I'm also interested in Mac, and Windows builds.
Especially setting them up on Appveyor, and Travis /Mac.

Translating on windows is the document which describes the windows builds. Not sure my cheap windows laptop has enough memory for it. It's only got 4GB of ram, and I expect it's going to take hours to compile.

The buildbot contains Nightly builds for windows as well as other platforms. So I might try using them to debug stuff, and I guess do extension development. Failing that, maybe I can copy a build off someones computer!

Hopefully I won't need to hire some remote servers for compiling stuff. It doesn't seem like I'd need to be recompiling all of pypy each time for extension development.

Thursday, March 08, 2018

pygameweb - 0.0.4 - Northern Rockhopper Penguin

pygameweb is the source code for the pygame website at

Northern Rockhopper Penguin (photo by Brian Gratwicke)
  • #52 Adding ability to deploy through travisci. And docs.
  • #46 Wiki readability. Wiki tools on left out of reading line.
  • Twitter card for releases. For nicer view when posting to twitter.
  • Wiki table of contents improvements.
  • builds docs for pygame, when something lands on pygame github master branch.
  • project release feeds bugfix.
  • Only show one recent release for each project. Some projects release a few times in a row.
  • Wiki, nicer pre code handling.
  • Wiki code uses inline colors, so copy/paste works with colors too.
  • use https for login and register buttons, even when on http (see Why can't we turn off HTTPS?).
  • Ask more bad robot web crawlers not to be bad.
See a full diff of changes since last release.

Tuesday, March 06, 2018

On the threshold of a journey with a Jedi, a Sphinx, and a Homebrew.

Feels like practice.
It all started some days ago, late at night,
with a comment made by someone on the internet...
"Threshold does not take keyword arguments"
As is often the case when you want to correct
something on the internet,
this comment lead me on a journey
(not one where I sell numpy for 22 million monies,
a different type of journey
involving text editors and compilers,
and lots of failing tests).
Software archeology -- not with Indiana,
but at least there was a Jedi and a Sphinx.

"Relax. Don't worry. And have a homebrew." -- Charlie Papazian

And with a warm brew in hand,
herein begins my tale...

Starting at the end, where all poor stories start,
with the comment I will send "TheBob427" (actual real nick),
right after I publish this blog post... [-1]

Dear TheBob427,

yes, this function doesn't take keyword arguments and you are right
that it's super buggy. However, now after some days of bashing my keyboard,
it should actually work. Additionally it should be slightly less weird.
It was a fun journey. I guess you'll have to wait for the next release to use it.
Which should be here 'soon'. Which in open source minutes is 
approximately the same amount of time as 'when it's done'.

yours sincerely,

ps. you could also use PixelArray.replace instead.

So, I updated the pygame.transform.threshold function, tests and docs [0].
Apart from that function being buggy and super confusing...

It was sort of a test of how we could improve all the pygame functions for pygame 3000.
Whilst it still doesn't follow the new pygame C API yet[1],
I wanted to try a few things like 'new' C99/C11 features
(thanks ffmpeg for tricking MS into dragging their compiler into the 90s),
new docs features (param types, and examples) and such like.

It's quite a bit of C and python test changes,
and took me quite a few days longer than I expected.
Simplifying here. Simplifying there. Simplifying everywhere.

One thing it does now is use sphinx type parameters.
Apart from the docs being improved, tools like
jedi can use them for autocomplete and type inference.

So, now editors like vim, sublime, and vscode can autocomplete with pygame.
(I had to change a few things for the 'odd' but 'cool' pygame
'missing module' imports feature so the type inference worked in jedi [2])


For examples, sphinx can now easily include source code.
So we can include pygame.examples code, or even test cases.
The 'nulledge' python search code search doesn't seem all that good
for many pygame things, and it lacks https. So i'm thinking of dropping
that and manually linking to relevant examples/test code.

Additionally, we can link to github source code, and even
the documentation files themselves. Which is handy for
'edit this'/'fork on github' style buttons.

Feels like practice.

[-1] a comment,
[0] threshold function pr,
[1] pygame C api issue,
[2] jedi confused with pygame pr,

Thursday, March 01, 2018

Pre release pygame builds through pypi and pip.

(If there's any python packaging experts reading this, please comment if this doesn't seem reasonable. You will get 42 internet points, and a cookie if you ever visit Berlin).

We want to be able to more widely test pre release builds, to avoid bugs and try things out on different OS/hardware combinations. This is the plan on how it will be done for pygame.


Pre release 'dev' and release candidate('rc') builds of pygame through pip and pypi.
  • every merge into master releases a dev build. 1.9.4.dev1+git.77.facebeef123
  • every release candidate tagged releases an rc build. 1.9.4.rc1

How to install pre release builds.

Pip pre-release documentation
`pip install pygame --pre`
Requiring a specific development build is possible according to the documentation.
`pip install pygame==1.9.4.dev1`

Version naming.

From packaging guide pre release versioning:
    zero or more dev releases (denoted with a “.devN” suffix)
    zero or more alpha releases (denoted with a “.aN” suffix)
    zero or more beta releases (denoted with a “.bN” suffix)
    zero or more release candidates (denoted with a “.rcN” suffix)
  • Dev versions will be named like this: 1.9.4.dev1+git.77.facebeef123 (version+local_version). With the dev1 becoming dev2, dev3. git describe can be used to get this info.
  • pygame.version.rev should have the git hash
  • Uploading wheels currently only happens when a tag happens. (eg, 1.9.3). Dev builds would instead need to happen when something is merged into master.
  • release candidates can be done with a tag. 1.9.4.rc1 will be release candidate 1.

git describe, for getting commits since last release.

Can be used to get the number of commits since the last release.
git describe --tags
Using this we can grab the info we need to build the pypi-happy version strings.
Note, it needs to use --tags if we don't have annotated tags. (which we should use going forwards).

Dev builds to test pypi instead?

  • piwheels don't download from testpypi, so we need to release to pypi to get these.
  • Also, the packaging guide doesn't talk about doing this. (it's just used for testing, and the DB is wiped occasionally).
  • Will pypi be cluttered with lots of dev releases? 3-20 per month (* 30 files)? I don't think this is a problem. There's nothing in the packaging guide to say don't do this.
  • people may have a need to rely on a specific dev version (especially since pygame is released infrequently), and testpypi gets wiped.

Sunday, February 25, 2018

Why can't we turn off HTTP?

Currently, I'm sitting in a library not able to access HTTPS (probably... I guess it depends when you're reading this) for a particular website.

I like to spend time in libraries. Especially libraries in exotic far off places. Surrounded by book-smell, and curious people learning. Unfortunately some libraries have shockingly bad internet. Some of them decide they will filter which websites people can look at so that they are safe from viruses and nasty stuff which damages their computers.

It's exciting that 60-70% of internet connections to some websites are encrypted these days. But why not turn HTTP off and force everyone to use HTTPS?
Indeed. Why not turn off HTTP completely? I'll try to answer that here (for; your website may be different).

Unfortunately these filters in libraries are sometimes pretty terrible and ancient. And when you travel, and spend time in libraries and use hotel internet you begin to notice that the internet that you know at home or work is kind of not the same all around the world. A shocking revelation I know things are different in different places. Soon there will be research funded to confirm this for you in ten years.

One thing they do is have proxy servers that jump in the middle between the browser and the server. Sometimes in strange ways that don't quite work.

Oh. So this is why I can't internet-all-the-things whilst sitting in a hammock on a small island.

Universities are another type of place I've had bespoke-internet-experiences.

So, this is one reason I like to still offer things over normal HTTP, because some people are blocked from HTTPS. A person can still choose already to use HTTPS (which technically still does have a MITM attacks going on from university/corp/country proxies and the like). We could also use HSTS ( which is the proper solution rather than redirects, but that means HTTP doesn't work. Because leaving HTTP on means it's possible HTTPS can be worked around.

If you're privileged enough to have good internet then, please take our EFF friends advice and use the https-everywhere addon for your browser. Unfortunately some things will still break for you.

However, could we still do better? And still make it accessible for those people who are stuck with horrible internet connections. Yes, of course!

So, now URLs permanently redirect to, and the login URLs even on non-HTTPS point to HTTPS. Additionally, I've updated lots of links in the docs and on the internet to HTTPS URLs, and will continue doing that where possible. Search engines always link to HTTPS (if available) it seems, and adding canonical HTTPS link tags in the headers is being tracked here: #34

I would like to update hundreds of mixed HTTP/HTTPS documents to turn all the HTTP links to new HTTPS links... but instead I chose life. Is there a header for that? Yes there is... pop on over to and learn all about it.

Browsers are now rolling out HTTPS upgrade headers, where they send a hint that they can go to a HTTPS URL when one is available.
You can also make a content security policy which allows the browser to upgrade old unecrypted links to HTTPS ones. You're telling the browser "Dear browser, everything should work with HTTPS, but I haven't updated all the URLs yet". Seems to be at 77% of browser support at the moment, and even the latest version of Edge supports it.

But remember, even if a browser supports it, it doesn't mean it hasn't been configured to use a proxy in the middle which has issues with HTTPS. Or that 500ms latency turns those several extra back-and-forths HTTPS requires into an unbearable thing to use. With three 500ms round trips, that turns into more than 1500ms. Compared to 20ms*3==60ms for someone on a fast connection nearby.

The reason we can't have zero round trips with HTTPS everywhere I hear you ask? That should would be good for people in far off lands with bad internet connections. Let me tell you my friend those same proxies, and Idiot of Things with ancient buggy HTTPS stacks. Blame your old printer.
But I've enabled 0-rtt and HTTP/2 anyway, and the reason is that those same broken internet connects are going to be broken anyway. Might as well make it slightly faster and safer for everyone else, and the broken connections can use HTTP which they would have to do anyway.

Is TLS fast Yet? Yes. Sort of.

With these changes, there's been a 30%(handwavy-made-up-stat) increase in HTTPS connections, and now the majority of connections are using HTTPS. Hopefully as more links are updated (18 years of links on the interwebs might not ever be upgraded).
Cloudflare (which is using for HTTPS) supports HTTPS on some pretty ancient computers (we are paying for the pro plan). Even my android 2 phone works with it. is still open for business fun on HTTP however, even though most people probably won't go there 'soon'.  The people in far off places with weird internet connections, and those retro computing people with the browser user agent string "AmigaVoyager/3.2 (AmigaOS/MC680x0)" will still be able to download their files. is //retro computer friendly// after all. But we're also friendly to people in far off lands sitting in hammocks and to people who like book-smells + learning.

Thanks for reading.

Saturday, February 24, 2018

#RaspberryPi binary #pygame wheels for pip rolling into a #python cheese shop near you.

It seems there's some sort of giant build cluster for Raspberry Pi that builds binary wheels (python package files, that don't need to be compiled). Automatically. We live in the future.

Feels like progress.

Now it's compiling pygame nicely. So `pip install pygame` works with the current release on Raspbian now (the Debian distribution put out by some Raspberry Pi people). Thanks to Ben Nuttall for that!
These are built automatically when new things are released to pypi. So the pygame 1.9.4 release should go on there without us having to do anything.

ps. Anyone reading this with a binary package they maintain... you can just ask them to install the Debian dependencies... or submit a PR to do so.
pps. Speaking of binary wheels. The other day I submitted a PR to which is python opencv packaged up nicely for pip. It should be much easier to do computer vision things that you can actually distribute to others (who aren't fond of C++ compilation fun).

Friday, February 23, 2018

Mu, the little #python editor that could.

Nicholas and the Mu gang have been busy with their python editor, and it now supports pgzero.

Mu is a simple code editor for beginner programmers.

I'm pretty excited about this. Here's a little demo of it in action with pgzero...

Whilst it's still in 'beta' with rough edges... it's pretty neat already. Also, they seem to be quickly moving through issues people in the community encounter with each beta release.

It's been super-lovely to see the python community get behind them and offer their support.

Stop idling, and go have a look:

pygame documentation updates

There are some documentation updates on

The website documentation builder was waiting for updates from bitbucket(our previous code host). lol? oops. Robots running in the cloud doing things no one wants them to do anymore. I had to write a new github integration, so now commits to master on github/pygame/pygame will cause the website docs to be rebuilt automatically again.

And python -m didn't work with the wheel builds... because they don't include docs (for some reason). So now, if the docs can't be found locally, it opens the web browser instead.
There were a bunch of documentation updates from Ian Mallett and Lenard Lindstrom which are finally up on the website. Lots of editing, and improvements to the tutorials.
Additionally a bunch of old links were fixed. Mostly to point to https:// versions of pages.
The docs are being built with a new version of Sphinx, which has nicer output in a few ways. See

Since it was a major version upgrade of Sphinx, there were some small expected issues to deal with, including their python API being broken/deprecated (just call the command line program instead), and disabling the on-by-default smart quotes(which are terrible for people copy pasting code examples).

Also, the launchpad PPA is building again. It got stuck because someone snuck a gpg field in their git commit, which broke the bzr mirror code. Except the packaging for that ppa is from 2013... and needs updating. Luckily it seems a few people have done the packaging work in various flavours of Debian... but somehow none of them have gone through. But anyway... The badge is getting made again(displayed on the readme) with the github webhook, and will alert us if things break on ubuntu/i386/amd64/arm7f.

Speaking of badges... there's now a 'commits since last release' badge on the Which links to a diff between the last release.

Finally, I updated all the old bitbucket wiki pages to point to the same link on the pygame wiki ( eg, points to

Thursday, February 15, 2018

Hey! It's work on pygame stuff week.

So, it's been about 9 days since I had "a fun day working on pygame stuff". That morning I woke up, and just started working on pygame things. This is a pattern with me between big freelance contracts. Last year I spent some months on pygame stuff, and also the year before that some months.

It was such a fun day... I just kept going. And here we are 9 days later with a web log of some of the things that happened.
A fun 9 days working on pygame stuff.

New website changes.

Got a new version of the pygame website out. It took a couple of days, but I fixed a number of issues, and improved a few things. In the process I found a lot more issues on the website than I fixed. So there are now more issues listed than when I started. Feels like progress.

"it's the schrödinger's bugtracker" -- þeshipu

pygameweb 0.0.1 - Weedy Seadragon

Witness the beauty of the Weedy Seadragon (photo by

#30 https url scheme default(not in DEBUG). https login links even on http.
#29 News feeds (rss, atom) are in the header of every page again.
#28 Improved table of contents on the wiki. Syntax help on edit page. Clear cache after edit.
#27 Changes are now shown again when people do a new project release. Project test improvements.
#31 /admin turned off by default, APP_ADMIN config variable.
#13 Admin interface improvements

"Are pygame releases going to have code names too? :-)" I guess so, but maybe the code name for all of them should be "Weedy Seadragon", because it's such a magical creature.

Debian bugs.

I've investigated a number of Debian reported issues, and even managed to fix some of them. It never ceases to amaze me that testing on other systems uncovers real bugs which also happen on other platforms (but in harder to detect ways usually). So, for me, it's always worth working on weird platforms - if only for this reason.

Thanks to Dominik for bringing them into the pygame issue tracker, and for improving the Debian pygame package generally.

It's important to get these fixes in, as I think they are blocking a Debian release of pygame 1.9.3. ... which also blocks derivatives such as Raspberry Pi rasperian distro (who don't provide any support to pygame, or Debian, and just take from Debian packages mostly). Another important derivative is Ubuntu, which also hasn't updated their pygame package.

Here is the Debian tracker for the pygame package, which lists things that need to be done. If anyone can spot something I could help out on there, please feel free to point it out - as I have some days now to investigate and fix things.

Here is the build status on the various Debian platforms:

Thanks to the OpenPOWER foundation and the University of Campinas who provide Power virtual machines to open source developers. It still feels magical to me when I log into a box on another continent to shell-about.

So, I got some Power PC virtual machines for testing. I fixed a few pygame on Debian bugs, and wrote on their issues telling them they are fixed now. Fingers crossed a newer pygame package will make it into Debian, Ubuntu and Raspberian.

Outdated installation instructions from around the net.

TLDR; If you see anywhere on the net that has outdated pygame install/compile instructions, please let them know, and send them a link to GettingStarted.

One thing I've noticed is that there are quite a few web pages on the interwebs describing 'how to install pygame'. Which were all helpful in the past, but are now out of date. The pygame website itself is one place with outdated instructions too. Now they just cause people to fail to install old versions of pygame, and for them to report the same old bugs again and again.

I'm asking people to:
  1. Link to the Getting Started guide, which should have tested installation methods for many platform/python combinations:
  2. Update instructions that tell people to install from the outdated bitbucket repository (we moved to github).
I've been updating the pygame website itself to follow this advice in parts where it wasn't current.
Also soon, the bitbucket repo will get a message added to it:
In the coming months I'll make that bitbucket repo error, with the same instructions.

Additionally, I'll add a message on compile failure to link to the relevant Compile* page.
Eg, if there's a failure on a Ubuntu machine it will link to the compile page for Ubuntu.

For all the compile pages, they will be updated to add the versions they were confirmed working (luckily, this info is sort of stored in the wiki version history). Old versions of Ubuntu require different compilation advice to the latest Ubuntu versions. Latest version instructions will be up the top of the page, and old versions underneath. I'll also gather links from around the internet, because sometimes better compilation instructions are listed on peoples blog posts than in the wiki. All the Compile pages are listed here: Further development/contributing instructions are listed here: Hopefully giving the Compile* pages some more structure, they will become better in the future.

Python 3.7.0b1

Some python 3.7.0b1 issues were fixed, and I got it working on Travis CI with linux. There is still work to try it out on Windows, and Mac, and to make binary wheels for for all the platforms.

Are we PyPy yet?

It seems various people have been working on making the PyPy CPython C API handling improve a lot. As well as fixing issues here and there with pygame when compiled with pypy.

Wow. Many thanks to all the people working on that.

Now it's at a stage where a bunch of things actually work. Not enough to play a game on it... but getting much closer.

Learning this good news... I got PyPy building on Travis CI/pygame/pygame (with some tests still failing). Then started a milestone for "Are we PyPy yet?". Stuart Axon has already filed one issue, and asked PyPy developers for help on how to fix it. Stuart wasn't joking saying PyPy devs are super helpful.

This one particular issue has to do with surface locking and reference counting. In CPython when you do "del array" it does what you think... and deletes the array right then and there. However, PyPy has a different type of memory management, and PyPy will decide when to delete things thank you very much!

Like with files in python, it was suggested we gain a method like '.close()' to explicitly deal with these locks. Instead of relying on reference counting semantics. Also, to use a context manager for with.

PixelArray is using Surface locks. It does this so if your Surface is graphics hardware memory underneath your operating system won't crash on you (I hope).
Leave the pixels alone!!!! What did they ever do to you?!?

Next up is to start filing pygame issues for each of the test failures and then slowly fixing them. Some of them should be fairly easy, and others are downright scary (BufferProxy and no ctypes.pythonapi on PyPy makes me afraid... very afraid).

Then we have to figure out how to build binary wheels for the various platforms, and probably many other things... All PyPy related work will be tracked on the Are we PyPy yet? milestone in the pygame issue tracker. Not sure how long this work will take, so this won't be included in the upcoming 1.9.4 release I don't think.

The mysteriously weird 'moiré pattern'.

I was clicking-about on stack overflow and came across this weird thing in popular question there. This strange thing happens when drawing thick circles with pygame.draw.

Before: moiré pattern sort of looks nice

After: just a boring filled bit of red. Nothing interesting. Move along.
That was sort of a fun one to hack a fix for.

I love Pulseaudio. And sarcasm.

I have much respect for the pulseaudio developers... linux audio is hard work.

There's a couple of issues with sound and pygame on linux with the 'manylinux' binary wheels we have. It took me some time to track the causes down, and find ways to reproduce the issues.

TLDR; midi music is broken with pygame installed by wheels.

One has to do with arch linux using a different configuration directory to every other linux of the last 20 years for timidity. And also to do with how we are not installing special config files on random parts of the linux file systems when people install the wheel with pip.

So, there are a few different software music synthesizers included with pygame. One is timidity (linked above) and the other is fluidsynth. The soundfont playing synth.

"Had enough with linux audio issues for today. I've got a sneaking feeling that pulseaudio and fluidsynth are to blame somehow for high cpu on linux systems when installing from pip wheels."

So, I just sort of paused at that point. I guess we need to disable timidity (or patch it for arch linux), and for the case when timidity isn't installed. Also, we need to handle soundfonts... either provide a message to the user if they are missing, or see if there are some FLOSS-happy ones we can distribute just on linux.

ps. During these investigations, I found out that fluidsynth has been saved from sourceforge and has had a whole bunch of releases. Awesome.

Updating old links again...

Another pleasant delight whilst searching-around-the-interwebs... I found a patch logged against an old mirror of the bitbucket repo on github. This had a few really good documentation updates, and an addition of a 'pitch_bend' method... which allows you to do pitch bends, as the name suggests. Luckily @endolith (a hardcore Electronics and DSP engineer doing interesting stuff) made a new PR, and so those improvements will finally make it out into the world. Somehow I overlooked it on bitbucket too.

Additionally, a couple of new issues were filed against out bitbucket repo... which I haven't closed down yet(because reasons). They are for a couple of SRC_ALPHA regressions it seems. I'll add them into github soon, and also then to the list of blockers for a 1.9.4 release.

1.9.4 release planning.

"About time for another release I guess?"
About 5 days ago I asked the mailing list if we could do a release, and that I intended to do a release. To give people time to mention issues, do testing, and let me know if they are still working on things to release that we should wait for.

Which is nice because people let me know about various issues they'd liked fixed and pointed me to the details (including the SRC_ALPHA issues above). I started looking around for things to merge in, looking at other bug trackers (like the Debian one).
Then, I started a checklist of things to do for a pygame release, and also started filing issues with the correct labels and using a milestone.

I want to get this work out so I can start merging in SDL2 stuff into the master branch. That's what I'm really looking forward to.

A new place to idle the day away chat.

We have a pygame discord server for anyone who wants to chat about pygame projects.

/me sheds a tear for irc.

Feels like progress. 

Every time I look at that bug list it gets bigger! Schrödinger has a lot to answer for.
Stop messing about on your phone and get to work!
It was a fun 9 days in floss. Gonna do more!