We highlight the exotic quantum criticality of quasi-two-dimensional single-component fermions at half filling that are minimally coupled to a dynamical Ising gauge theory. With the numerical matrix product state based infinite density matrix renormalization group method, we discover a robust quantum critical line in the infinite cylinder geometry, where gauge confinement and dimerized translation symmetry breaking emerge simultaneously. We investigate how the transition can be split by a Z2 topologically ordered dimerized phase that is stabilized by additional short-range repulsive interactions. We conjecture a u(1) deconfined criticality scenario, propose a corresponding low-energy effective field theory of the exotic quantum critical point in the two-dimensional limit, and identify its shortcomings.