#
A Treatment of Computational Precision, Number Representation, and Large Integers in an Introductory Fortran Course
Article

## William Richardson, Francis Marion University, United States

JCMST Volume 25, Number 3, ISSN 0731-9258 Publisher: Association for the Advancement of Computing in Education (AACE), Waynesville, NC USA

## Abstract

Computational precision is sometimes given short shrift in a first programming course. Treating this topic requires discussing integer and floating-point number representations and inaccuracies that may result from their use. An example of a moderately simple programming problem from elementary statistics was examined. It forced students to confront problems associated with number representations and investigate appropriate ways to circumvent them. In the example, integers were represented using two's complement form, and floating-point numbers according to the Institute of Electrical and Electronics Engineers (IEEE) standard 754, both using 32 bits. Floating-point number representation (64-bit) was necessary to achieve results accurate to two decimal positions. Finally, Fortran 90/95 subprograms were written to implement arbitrarily precise operations with large integers. Students in an introductory course can write subprograms for addition and subtraction of large integers. More sophisticated subprograms that depend on the fast Fourier transform were written to perform multiplication and division of large integers. These subprograms are more suited to an advanced programming or numerical analysis course. The subprograms written to perform large-integer operations were used to determine exact solutions for the statistical problem.

## Citation

Richardson, W. (2006). A Treatment of Computational Precision, Number Representation, and Large Integers in an Introductory Fortran Course. Journal of Computers in Mathematics and Science Teaching, 25(3), 213-237. Waynesville, NC USA: Association for the Advancement of Computing in Education (AACE). Retrieved December 4, 2022 from https://www.learntechlib.org/primary/p/6298/.

© 2006 Association for the Advancement of Computing in Education (AACE)

### Keywords

## References

View References & Citations Map- Burley, C. (1998). User notes on Fortran programming (An open cooperative practical guide). Comparison between Fortran and C. (chap. 1, sect. 2). Retrieved February 19, 2006, from http://www.ibiblio.org/pub/languages/fortran/
- Cambridge University High Performance Computing Facility. (2000). The case against C. Retrieved February 19, 2006, from http://www.hpcf.cam.ac.uk/C_rant.html
- Cooley, J. W., & Tukey, J. W. (1965). An algorithm for the machine calculation of complex fourier series. Mathematics of Computation, 19(90), 297-301. Cooley, J. W. (1990). How the FFT gained acceptance. In S. G. Nash (Ed.), A history of scientiﬁ c computing (pp. 133-140). Reading, MA: Addison-Wesley. ISBN 0-201-50814-1
- Crandall, R., & Pomerance, C. (2002). Prime numbers: A computational perspective. New York: Springer.
- Decyk, V., Norton, C., & Szymanski, B. (1995). Introduction to object-oriented concepts using Fortran 90. Retrieved February 19, 2006, from http://www.cs .rpi.edu/~szymansk/OOF90/F90_Objects.html
- Hayes, B. (2003). A lucid interval. American Scientist, 91(6), 484-488. Heideman, M. T., Johnson, D. H., & Burrus, C. S. (1984). Gauss and the history of the fast Fourier transform. IEEE ASSP Magazine, 1, 14-21. Hewlett-Packard Development Company. (2004). HP-UX ﬂ oating-point guide: HP 9000 computers. Retrieved February 26, 2005, from http://docs.hp.com/en/B3906-90006/go01.html#d0e21238 And http://docs.hp.com/en/B3906-90006/go01.html#d0e21154
- Knuth, D.E. (1981). The art of computer programming, seminumerical algorithms, Vol. 2 (2nd ed.). Reading, MA: Addison-Wesley.
- Nyhoff, L.R., & Leestma, S.C. (1997). Fortran 90 for scientists and engineers. Upper Saddle River, NJ: Prentice-Hall.
- Prentice, J. (1995), Fortran 90 for science students. Retrieved February 19, 2006, from http://www.lahey.com/PRENTICE.HTM

These references have been extracted automatically and may have some errors. Signed in users can suggest corrections to these mistakes.

Suggest Corrections to References