Abstract: During the service lives of ballasted tracks, the ballast experiences degradation, such as breakage and fragmentation, which reduces the stability of the tracks. Ballast breakage is directly related to the shapes, sizes, and stress states of particles. The key to solving this problem is to determine the breakage parameters of railway ballast. In this study, through uniaxial compression and uniaxial shear tests, the factors associated with ballast breakage for a simple stress state were obtained. Then a refined discrete element model of railway ballast was established. The Box-Behnken method was used to design a simulation test, and a response surface method was used to obtain the optimal ballast breakage parameters. Lastly, the results of a direct shear test were compared with the simulation results to verify the correctness of the parameters and to study the shear resistance of the ballast aggregates. The results showed that for ballast particles with sizes of 22.4–63 mm, the characteristic stresses of compression and shear were not significantly affected by the size of the particles, but were greatly affected by their shape. In particular, long particles were more likely to break. During the direct shear test, the cumulative ballast breakage ratio gradually increased to 13.97%. The optimal breakage parameters of the ballast determined by this approach have high application value in the management of ballasted tracks.