Williams’ effort condemned the sorry Sulphurites to a fifth straight home defeat, which represents their worst sequence as an EFL outfit.
Away keeper Harvey Davies had earlier made an excellent save to push Matty Daly’s rising 12-yard drive around his far post.
Chris Long narrowly missed the target from the edge of the box at the other end before Williams opened the scoring just past the half-hour mark.
Mickey Demetriou’s long throw from the left was allowed to bounce in the home box and the 19-year-old midfielder headed in off the far post from two yards out.
After the break, Joel Tabiner survived strong home appeals for a penalty when he appeared to pin Daly to the ground, while Harrogate midfielder Levi Sutton headed wide of an empty goal after Davies had punched away a George Thomson corner.
Centre-back Connor O’Riordan might have made the margin of victory greater, though, when he shot wide from eight yards and substitute keeper Lewis Thomas, making his EFL debut as a replacement for Mark Oxley, also made a smart stoppage-time save to deny Courtney Baker-Richardson.