The Z axis is usually used for depth, so it’s going to be perpendicular to whatever your frame of reference (i.e. projection plane) is.
If it’s upright in space, like a computer screen, the Z axis will be horizontal. If it’s a sheet of paper on a desk, then yes, I suppose it could be argued to be vertical instead.