The ones used in early computers were acoustic delay lines, which consisted of a transducer (speaker) which sent sound signals into a channel filled with a suitable material (mercury was often used), with another transducer (acting as a microphone) at the other end. (There are other ways to produce delay lines, but the ones used in computer memories generally used the acoustic approach.)
A string of bits could be stored in the delay line by taking its output, sending it through some electronics (including an amplifier) to 'clean up' the signal, and then directing it back to the start of the delay line for re-insertion.
Although they were cheap and simple, they had one large drawback; they were not random access. If the computer needed a value that had just been sent into the delay line, it had to wait until it emerged at the other end. This made alternatives such as Williams tube storage attractive, despite their issues.
Once core memory appeared, delay line memories rapidly disappeared.