Controlling the directionality of spin waves is a key ingredient in wave-based computing methods such as magnonics. In this Letter, we demonstrate this particular aspect by using an all-optical pointlike source of continuous spin waves based on frequency comb rapid demagnetization. The emitted spin waves contain a range of k vectors and by detuning the applied magnetic field slightly off the ferromagnetic resonance (FMR), we observe X-shaped caustic spin wave patterns at 70 degrees propagation angles as predicted by theory. When the harmonic of the light source approaches the FMR, the caustic pattern gives way to uniaxial spin wave propagation perpendicular to the in-plane component of the applied field. This field-controlled propagation pattern and directionality of optically emitted short-wavelength spin waves provide additional degrees of freedom when designing magnonic devices.