TT Lab
Get started
Learn Learning paths Courses

Shaders and the GPU Pipeline

Paste a Pattern and Pull Out Samples

Continue in TT Lab

Goal

Make a texture, implement two sampling methods, and see the difference of perspective-correct interpolation with your own eyes. At the end, you confirm in numbers the shimmering that arises from minification and how mipmaps remove it.

Why it matters

If you attach a color to each vertex, a triangle has only three colors. A texture takes the color off the geometry and puts it in a separate image, and only decides where in that image each point of the surface points. That coordinate is UV, and the rasterizer interpolates it per pixel.

Two things are especially important here. One is perspective correction. If you simply blend UV in screen coordinates, the pattern is warped on a surface with perspective. This is because even intervals on the screen are not even in 3D space. The other is minification. If one pixel covers dozens of texels, picking just one of them cannot represent the pattern, and even if the camera moves slightly, the screen shimmers. The mipmap is the answer to that.

Steps

  1. Put the toolbox in /root/texture.
  2. /root/texture/tex.py and out/checker.png — the checker pattern.
  3. /root/texture/sample.py — nearest-neighbor sampling.
  4. /root/texture/bilinear.py — bilinear sampling.
  5. /root/texture/out/affine.png — interpolation without correction.
  6. /root/texture/out/correct.png and diff.png — perspective correction.
  7. /root/texture/out/mip0.png, mip1.png, out/07-mip.txt — minification and mipmaps.

Notes

Put the drawing toolbox in place

Save /root/texture/gfxlib.py exactly as in the example, and use /root/texture/check.py to draw a test pattern and make /root/texture/out/00-check.png. The pattern is a 64x64 black background with a white (255,255,255) diagonal line from (0,0) to (63,63), and over it a red (255,0,0) horizontal line from (0,32) to (63,32).

From this lab on, you do not rebuild the PNG encoder. We hand you the same code you built by hand in the first lab as a tool — because file formats are not what you learn here.

The lab Pod has no volume, so the files you made in the previous lab are not kept. That is why each lab starts by putting the toolbox in place again.

Create Canvas(w, h, bg), draw the two lines with line(x0, y0, x1, y1, rgb), and then save with write_png(path). Draw the horizontal line later, so that the intersection (32,32) becomes red.

In this lab you use it to export the texture as a picture and produce the pattern-mapped quadrilateral.

A checker pattern texture

In /root/texture/tex.py, define SIZE = 64, CELL = 8, LIGHT = (240,240,240) and DARK = (40,40,60), and make a TEX list holding 64x64 texels in row-major order and a texel(i, j) function. The color of a cell is LIGHT if ((i//CELL) + (j//CELL)) % 2 == 0 and DARK otherwise, and texel wraps the indexes with % SIZE. Then export it as /root/texture/out/checker.png (64x64).

It is row-major, so TEX[j * SIZE + i] is the texel (i, j). It is the same rule as the pixel layout in the previous course.

texel wrapping the indexes is the repeat wrap mode. If you decide it once here, the sampling functions do not have to worry about the range.

The reason for choosing a checker pattern is that it is the pattern in which warping and shimmering show up best. That is why it has long been used as a test image for textures.

With 8-pixel cells on 64x64, there are eight cells in each direction.

Nearest-neighbor sampling

In /root/texture/sample.py, make sample_nearest(u, v). Find i = floor(u * SIZE) and j = floor(v * SIZE) as integers and return texel(i, j). UV outside the range is handled by the wrapping in texel.

Use math.floor and convert with int. If you use only int() on negative UV, it truncates toward 0 and you are off by one cell — int(-0.3) is 0 but floor(-0.3) is -1.

The wrapping is done by % SIZE inside texel. Python's % gives a positive remainder for negative numbers too, so it becomes the repeat mode as it is.

u = 1.5 should give the same result as u = 0.5. That means the pattern repeats.

Bilinear sampling

In /root/texture/bilinear.py, make sample_bilinear(u, v). After shifting with x = u*SIZE - 0.5 and y = v*SIZE - 0.5, find i0 = floor(x) and fx = x - i0, and return three real numbers that blend the four texels (i0,j0), (i0+1,j0), (i0,j0+1) and (i0+1,j0+1) in the ratios fx and fy.

Subtracting 0.5 is the key. It is because the center of a texel is at (i+0.5)/SIZE. If you leave out this half cell, the picture shifts by half a texel, and it is hard to notice until you zoom in.

The blending expression is like this.

c00*(1-fx)*(1-fy) + c10*fx*(1-fy) + c01*(1-fx)*fy + c11*fx*fy

The four weights sum to 1, so if all four texels are the same color, that color comes out as is. You can check your implementation with this.

At a UV placed exactly at a texel center (for example 0.5/SIZE), the texel's color should come out as is.

Paste it without correction

Use /root/texture/quad.py to paste the pattern on a quadrilateral and make /root/texture/out/affine.png (256x256, black background). The four vertices, in the order screen coordinates, w and uv, are (30,40)/1.0/(0,0), (226,90)/4.0/(1,0), (226,166)/4.0/(1,1) and (30,216)/1.0/(0,1), and the triangles are (0,1,2) and (0,2,3). Interpolate uv simply with barycentric coordinates and pull out the color with nearest-neighbor sampling.

You can take w as the distance from the camera. The two right-hand vertices having a w of 4 means that side is far away.

In this step you deliberately ignore that fact and blend uv evenly on the screen. That is why the pattern is warped. The floor patterns wobbling in early console games was exactly this.

It is divided into two triangles, so the direction of the warp bends along the diagonal boundary. That bend is the characteristic symptom of this approach.

The barycentric coordinates and the bounding box scan are the same as what you built in the previous course.

Perspective-correct interpolation

Draw the same quadrilateral again by interpolating u/w, v/w and 1/w and dividing the first two by the last, to make /root/texture/out/correct.png, and export the pixel difference of the two pictures, amplified three times, as /root/texture/out/diff.png.

The correction formula is like this.

iw = b0/w0 + b1/w1 + b2/w2
u  = (b0*u0/w0 + b1*u1/w1 + b2*u2/w2) / iw

Only what you interpolate changes; the interpolation itself stays the same. The point is that the quantities that are linear in screen coordinates are not uv but u/w and 1/w.

For the difference image, compute min(255, abs(a - b) * 3) for each channel. The reason for amplifying three times is that a plain subtraction is dark and hard to see.

Look at the two pictures side by side. In the uncorrected one the pattern bends at the diagonal boundary, and in the corrected one you get the natural look of getting denser as it recedes.

Minification and mipmaps

Use /root/texture/mip.py to make a 64x64 checker with one-texel cells (LIGHT if (i+j)%2==0, otherwise DARK), and make one 32x32 mip level with a 2x2 average. Sample each of the two textures with nearest-neighbor on a 128x128 screen with u = (x+0.5)/128*5 and v = (y+0.5)/128*5 to make /root/texture/out/mip0.png and mip1.png, and write the mean absolute channel difference of horizontally neighboring pixels to /root/texture/out/07-mip.txt as energy0=, energy1= and level1_uniform=yes|no.

A checker with one-texel cells is the highest-frequency pattern. If you take the 2x2 average just once, two bright texels and two dark texels are mixed and every texel becomes the same color. So level1_uniform comes out yes.

You count the energy like this — for every y and x=0..126, add the three channels of |픽셀(x+1,y) - 픽셀(x,y)| (the placeholders are the pixel values) and divide the total by (count × 3).

energy0 comes out over 90 and energy1 comes out exactly 0. That difference is the size of the shimmering. If the camera moves even slightly, the pattern in the energy0 picture changes entirely, but the energy1 one stays as it is.

This stability is what you buy with the one third more memory the mipmap uses.