We present a detailed investigation into the intrinsic tunnel barrier reliability in AlOx-based magnetic tunnel junctions (MTJs) as a function of aluminum thickness and oxidation time. The intrinsic reliability is measured as the ramped breakdown voltage (V-bd) at room temperature for both positive and negative polarity. We find that Vbd generally increases with the resistance-area (RA) product of the MTJ. While this dependence is quite strong at low RA, it gradually weakens for higher RA. At fixed RA, Vbd also depends on the original Al film thickness with better properties for thicker Al. Finally, we observe a polarity dependence of V-bd which changes sign as the MTJ goes from thin Al to thick Al. We attribute the polarity dependence to the different quality of the top and bottom interfaces and conclude that the interface emitting the tunneling electrons primarily governs the barrier reliability.