Mechanical Engineering Question:

The total process of a the refining business starts at the oil field or gas field and runs all the way to the sending of processed hydrocarbon to a final user.
Upstream applies to the operation of exploration, drilling, hydrocarbon production, and transmission via truck, rail or ship or pipeline to the refinery intake valve.

Downstream includes all work done at the refinery, distillation, cracking, reforming, blending storage, mixing and shipping.
The case of heavy oil processing (oil sands etc.) and gas plant operation tend to cross the boundaries somewhat. Most are regarded as upstream operations even though downstream type operations are part of the processes. The production of chemical side products at gas plants (e.g. sulfur) is not generally segregated as a "Chemical Plant" operation.
Additional hydrocarbon production operations such as saddle plants, which remove a component from pipeline gas, are generally lumped with upstream.