A few quotes from the books, movies, and JK Rowling herself that hinted at Grindelwald's past with Dumbledore:
"Neither Dumbledore nor Grindelwald ever seems to have referred to this brief boyhood friendship in later life," Rowling wrote. "However, there can be no doubt that Dumbledore delayed, for some five years of turmoil, fatalities, and disappearances, his attack upon Gellert Grindelwald. Was it lingering affection for the man or fear of exposure as his once best friend that caused Dumbledore to hesitate?"