So, for example, the architecture of a CPU would include a high-level view of: the ALU; other computational elements (e.g. a floating point unit); the registers (both those visible to the programmer, and internal registers); the data paths which connect them all together; and the control system (usually microcode) which runs the whole ensemble.
The term is also used in a more restricted sense, to mean 'the user view of the system, across multiple models/versions'; in other words, it tries to separate out the implementation details from the time-invariant fundamental description.
In particular, modern systems make heavy use of modularization - breaking the overall system up into well-defined pieces (modules; or sub-systems, themselves composed of modules, in more complex systems): deciding where to put those boundaries, and defining the interactions across them, is a key part of a system's architecture (and of the job of the system architect, the person who lays out the overall design).
Another common technique is layering, which is an attempt to introduce an ordering among the sub-systems, to reduce/remove 'dependency loops' among the sub-systems. (Dependency loops are undesirable, as it makes the system more complex, and more bug-prone.)
If this can be achieved, it allows the system to be built more easily (as only the lowest-layer sub-systems need to be working at first), and more easily understood (since most of the higher-layered sub-systems can be ignored, when trying to understand the functioning of one particular sub-system).
The actual structure in most systems is usually a hierarchy, not a simple layered stack, as often more than one higher-level component will use the services of a given lower-level component.
Architecture is usually guided by defined goals/requirements that the system has to meet, but these are of only so much use. The problem is that any large system - and particularly software systems - have long lifetimes, and the environment, and uses, of the system usually change considerably over that time period - in ways that are usually impossible to foresee.
Hence the aphorism that "The hallmark of great architecture is not how well it does the things it was designed to do, but how well it does things it was never expected to handle." Longer lifetimes for an architecture are usually gained, not by focusing primarily on a system's goals and requirements, but by trying to understand, at a deep level, what the system does, and then organizing it to do that in as simple, general and flexible a way as possible.