We present an algebraic hp-multigrid method for high-order matrix-free methods. Algebraic multigrid methods often require information about matrix entries, which are not available in a matrix-free setting; however, when rediscretization for geometric multigrid is not available for a matrix-free method, coarsening must be constructed using information from the mesh. Leveraging only mesh adjacency information, this algorithm constructs an algebraic multigrid hierarchy without requiring geometric coarsening or explicit matrix assembly, making it well-suited for GPUĝ€'accelerated architectures. This paper presents the implementation of the matrix-free method in the high-fidelity computational fluid dynamics framework Neko, which utilizes spectral element methods with an implicit-explicit scheme to solve the incompressible Navier-Stokes equations. We utilize an hp-multigrid approach, where the problem is first coarsened from high-order polynomials to low-order polynomials, and then the low-order system is further coarsened spatially in an matrix-free fashion using mesh adjacency information. Finally, we present numerical results from the Dardel and LUMI supercomputers that demonstrate the performance and scalability of our method as well as its applicability to real-world applications.