In triangle ABC the sides satisfy a : b : c = 4 : 5 : 6. The cosine of the SMALLEST angle of the triangle is:

The smallest angle faces the shortest side, so it is A, opposite the side of 4 parts. The cosine rule gives cosA = $(b^2+c^2-a^2)/(2bc)=(25+36-16)/(2\times 5\times 6)=45/60=3/4$, so A ≈ 41.4°. Applying the same rule at the wrong vertex gives cosC = $(16+25-36)/40=1/8$ — that is the largest angle, about 82.8° — and cosB = 27/48 = 9/16 for the middle one. The bare ratio 5/6 is not a cosine of anything here.