Feed-forward visual processing suffices for coarse localization but fine-grained localization in an attention-demanding context needs feedback processing

English version

Autoři: Sang-Ah Yoo ^aff001; John K. Tsotsos ^aff002; Mazyar Fallah ^aff001
Působiště autorů: Department of Psychology, York University, Toronto, ON, Canada ^aff001; Centre for Vision Research, York University, Toronto, ON, Canada ^aff002; Active and Attentive Vision Laboratory, Department of Electrical Engineering and Computer Science, York University, Toronto, ON, Canada ^aff003; Visual Perception and Attention Laboratory, School of Kinesiology and Health Science, York University, Toronto, ON, Canada ^aff004
Vyšlo v časopise: PLoS ONE 14(9)
Kategorie: Research Article
prolekare.web.journal.doi_sk: https://doi.org/10.1371/journal.pone.0223166

Souhrn

It is well known that simple visual tasks, such as object detection or categorization, can be performed within a short period of time, suggesting the sufficiency of feed-forward visual processing. However, more complex visual tasks, such as fine-grained localization may require high-resolution information available at the early processing levels in the visual hierarchy. To access this information using a top-down approach, feedback processing would need to traverse several stages in the visual hierarchy and each step in this traversal takes processing time. In the present study, we compared the processing time required to complete object categorization and localization by varying presentation duration and complexity of natural scene stimuli. We hypothesized that performance would be asymptotic at shorter presentation durations when feed-forward processing suffices for visual tasks, whereas performance would gradually improve as images are presented longer if the tasks rely on feedback processing. In Experiment 1, where simple images were presented, both object categorization and localization performance sharply improved until 100 ms of presentation then it leveled off. These results are a replication of previously reported rapid categorization effects but they do not support the role of feedback processing in localization tasks, indicating that feed-forward processing enables coarse localization in relatively simple visual scenes. In Experiment 2, the same tasks were performed but more attention-demanding and ecologically valid images were used as stimuli. Unlike in Experiment 1, both object categorization performance and localization precision gradually improved as stimulus presentation duration became longer. This finding suggests that complex visual tasks that require visual scrutiny call for top-down feedback processing.

Klíčová slova:

Neurons – Vision – Flowers – Reaction time – Animal performance – Ellipses – Visual system – Neuronal tuning

Zdroje

1. Fabre-Thorpe M, Delorme A, Marlot C, Thorpe S. A limit to the speed of processing in ultra-rapid visual categorization of novel natural scenes. J Cogn Neurosci. 2001;13(2):171–80. 11244543

2. Keysers C, Xiao DK, Földiák P, Perrett DI. The speed of sight. J Cogn Neurosci. 2001;13(1):90–101. 11224911

3. Potter MC. Short-term conceptual memory for pictures. J Exp Psychol Hum Learn. 1976;2(5):509–22. 1003124

4. Potter MC, Wyble B, Hagmann CE, McCourt ES. Detecting meaning in RSVP at 13 ms per picture. Attention, Perception, Psychophys. 2014;76(2):270–9.

5. Rousselet GA, Macé MJ-M, Fabre-Thorpe M. Is it an animal? Is it a human face? Fast processing in upright and inverted natural scenes. J Vis. 2003 Jul 31;3(6):440–55. doi: 10.1167/3.6.5 12901715

6. Thorpe S, Fize D, Marlot C. Speed of processing in the human visual system. Nature. 1996;381(6582):520–2. doi: 10.1038/381520a0 8632824

7. VanRullen R, Koch C. Visual selective behavior can be triggered by a feed-forward process. J Cogn Neurosci. 2003;15(2):209–17. doi: 10.1162/089892903321208141 12676058

8. VanRullen R, Thorpe SJ. Is it a bird? Is it a plane? Ultra-rapid visual categorisation of natural and artifactual objects. Perception. 2001;30(6):655–68. doi: 10.1068/p3029 11464555

9. Grill-Spector K, Kanwisher N. Visual recognition. Psychol Sci. 2005 Feb 6;16(2):152–60. doi: 10.1111/j.0956-7976.2005.00796.x 15686582

10. Delorme A, Richard G, Fabre-Thorpe M. Ultra-rapid categorisation of natural scenes does not rely on colour cues: A study in monkeys and humans. Vision Res. 2000;40(16):2187–200. doi: 10.1016/s0042-6989(00)00083-3 10878280

11. Riesenhuber M, Poggio T. Hierarchical models of object recognition in cortex. Nat Neurosci. 1999 Nov;2(11):1019–25. doi: 10.1038/14819 10526343

12. Serre T, Oliva A, Poggio T. A feedforward architecture accounts for rapid categorization. Proc Natl Acad Sci. 2007;104(15):6424–9. doi: 10.1073/pnas.0700622104 17404214

13. Rousselet GA, Fabre-Thorpe M, Thorpe SJ. Parallel processing in high-level categorization of natural images. Nat Neurosci. 2002;5(7):629–30. doi: 10.1038/nn866 12032544

14. Hung CP, Kreiman G, Poggio T, DiCarlo JJ. Fast readout of object identity from macaque inferior temporal cortex. Science. 2005;310(5749):863–6. doi: 10.1126/science.1117593 16272124

15. Isik L, Meyers EM, Leibo JZ, Poggio T. The dynamics of invariant object recognition in the human visual system. J Neurophysiol. 2014;111(1):91–102. doi: 10.1152/jn.00394.2013 24089402

16. Liu H, Agam Y, Madsen JR, Kreiman G. Timing, timing, timing: Fast decoding of object information from intracranial field potentials in human visual cortex. Neuron. 2009;62(2):281–90. doi: 10.1016/j.neuron.2009.02.025 19409272

17. Kay KN, Winawer J, Mezer A, Wandell BA. Compressive spatial summation in human visual cortex. J Neurophysiol. 2013;110(2):481–94. doi: 10.1152/jn.00105.2013 23615546

18. Salin P-A, Bullier J. Corticocortical connections in the visual system: Structure and function. Physiol Rev. 1995;75(1):107–54. doi: 10.1152/physrev.1995.75.1.107 7831395

19. Tsotsos JK. A “complexity level” analysis of immediate vision. Int J Comput Vis. 1988;1(4):303–320.

20. Tsotsos JK, Culhane SM, Kei Wai WY, Lai Y, Davis N, Nuflo F. Modeling visual attention via selective tuning. Artif Intell. 1995;78(1–2):507–45.

21. Carlson T, Hogendoorn H, Fonteijn H, Verstraten FAJ. Spatial coding and invariance in object-selective cortex. Cortex. 2011;47(1):14–22. doi: 10.1016/j.cortex.2009.08.015 19833329

22. Cichy RM, Chen Y, Haynes JD. Encoding the identity and location of objects in human LOC. Neuroimage. 2011;54(3):2297–307. doi: 10.1016/j.neuroimage.2010.09.044 20869451

23. MacEvoy SP, Epstein RA. Position selectivity in scene- and object-responsive occipitotemporal regions. J Neurophysiol. 2007;98(4):2089–98. doi: 10.1152/jn.00438.2007 17652421

24. DiCarlo JJ, Maunsell JHR. Anterior inferotemporal neurons of monkeys engaged in object recognition can be highly sensitive to object retinal position. J Neurophysiol. 2003;89(6):3264–78. doi: 10.1152/jn.00358.2002 12783959

25. Grill-Spector K, Kushnir T, Edelman S, Avidan G, Itzchak Y, Malach R. Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron. 1999;24(1):187–203. doi: 10.1016/s0896-6273(00)80832-6 10677037

26. Hemond CC, Kanwisher NG, Op de Beeck HP. A preference for contralateral stimuli in human object- and face-selective cortex. PLoS One. 2007;2(6):3–7.

27. Niemeier M, Goltz HC, Kuchinad A, Tweed DB, Vilis T. A contralateral preference in the lateral occipital area: Sensory and attentional mechanisms. Cereb Cortex. 2005;15(3):325–31. doi: 10.1093/cercor/bhh134 15269109

28. Schwarzlose RF, Swisher JD, Dang S, Kanwisher N. The distribution of category and location information across object-selective regions in human visual cortex. Proc Natl Acad Sci. 2008;105(11):4447–52. doi: 10.1073/pnas.0800431105 18326624

29. Sayres R, Grill-Spector K. Relating retinotopic and object-selective responses in human lateral occipital cortex. J Neurophysiol. 2008;100(1):249–67. doi: 10.1152/jn.01383.2007 18463186

30. Kirchner H, Thorpe SJ. Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited. Vision Res. 2006;46(11):1762–76. doi: 10.1016/j.visres.2005.10.002 16289663

31. Crouzet SM, Kirchner H, Thorpe SJ. Fast saccades toward faces: Face detection in just 100 ms. J Vis. 2010;10(4):1–17.

32. Tsotsos JK. Analyzing vision at the complexity level. Behav Brain Sci. 1990;13(3):423–45.

33. Tsotsos JK. A computational perspective on visual attention. Cambridge: MIT Press; 2011.

34. Tsotsos JK, Rodríguez-Sánchez AJ, Rothenstein AL, Simine E. The different stages of visual recognition need different attentional binding strategies. Brain Res. 2008;1225:119–32. doi: 10.1016/j.brainres.2008.05.038 18585692

35. Boehler CN, Tsotsos JK, Schoenfeld MA, Heinze HJ, Hopf JM. The center-surround profile of the focus of attention arises from recurrent processing in visual cortex. Cereb Cortex. 2009;19(4):982–91. doi: 10.1093/cercor/bhn139 18755778

36. Hopf JM, Boehler CN, Schoenfeld MA, Heinze HJ, Tsotsos JK. The spatial profile of the focus of attention in visual search: Insights from MEG recordings. Vision Res. 2010;50(14):1312–20. doi: 10.1016/j.visres.2010.01.015 20117126

37. Hopf JM, Boehler CN, Luck SJ, Tsotsos JK, Heinze HJ, Schoenfeld MA. Direct neurophysiological evidence for spatial suppression surrounding the focus of attention in vision. Proc Natl Acad Sci. 2006;103(4):1053–8. doi: 10.1073/pnas.0507746103 16410356

38. Boehler CN, Schoenfeld MA, Heinze H-J, Hopf J-M. Rapid recurrent processing gates awareness in primary visual cortex. Proc Natl Acad Sci. 2008;105(25):8742–7. doi: 10.1073/pnas.0801999105 18550840

39. Wyatte D, Jilk DJ, O’Reilly RC. Early recurrent feedback facilitates visual object recognition under challenging conditions. Front Psychol. 2014;5:674. doi: 10.3389/fpsyg.2014.00674 25071647

40. Mohsenzadeh Y, Qin S, Cichy RM, Pantazis D. Ultra-rapid serial visual presentation reveals dynamics of feedforward and feedback processes in the ventral visual pathway. Elife. 2018;7:e36329. doi: 10.7554/eLife.36329 29927384

41. Evans KK, Treisman A. Perception of objects in natural scenes: Is it really attention free? J Exp Psychol Hum Percept Perform. 2005;31(6):1476–92. doi: 10.1037/0096-1523.31.6.1476 16366803

42. Moran J, Desimone R. Selective attention gates visual processing in the extrastriate cortex. Science. 1985;229(4715):782–4. doi: 10.1126/science.4023713 4023713

43. VanRullen R, Thorpe SJ. The time course of visual processing: From early perception to decision-making. J Cogn Neurosci. 2001;13(4):454–61. 11388919

44. Chen Y, Byrne P, Crawford JD. Time course of allocentric decay, egocentric decay, and allocentric-to-egocentric conversion in memory-guided reach. Neuropsychologia. 2011;49(1):49–60. doi: 10.1016/j.neuropsychologia.2010.10.031 21056048

45. Rothenstein AL, Rodríguez-Sánchez AJ, Simine E, Tsotsos JK. Visual feature binding within the Selective Tuning attention framework. Int J Pattern Recognit Artif Intell. 2008;22(5):861–81.

46. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, et al. Microsoft COCO: Common objects in context. In: ECCV. Springer; 2014. p. 740–55.

47. Brainard DH. The Psychophysics Toolbox. Spat Vis. 1997;10(4):433–6. 9176952

48. Pelli DG. The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spat Vis. 1997;10(4):437–42. 9176953

49. Praß M, Grimsen C, König M, Fahle M. Ultra rapid object categorization: Effects of level, animacy and context. PLoS One. 2013;8(6):2–11.

50. Walker S, Stafford P, Davis G. Ultra-rapid categorization requires visual attention: Scenes with multiple foreground objects. J Vis. 2008;8(4):21. doi: 10.1167/8.4.21 18484860

51. Davenport JL, Potter MC. Scene consistency in object and background perception. Psychol Sci. 2004;15(8):559–64. doi: 10.1111/j.0956-7976.2004.00719.x 15271002

52. Joubert OR, Fize D, Rousselet GA, Fabre-Thorpe M. Early interference of context congruence on object processing in rapid visual categorization of natural scenes. J Vis. 2009;8(13):11–11.

53. Akaike H. A new look at the statistical model identification. IEEE Trans Automat Contr. 1974;19(6):716–23.

54. Buffalo EA, Fries P, Landman R, Liang H, Desimone R. A backward progression of attentional effects in the ventral stream. Proc Natl Acad Sci. 2010;107(1):361–5. doi: 10.1073/pnas.0907658106 20007766

55. Cutzu F, Tsotsos JK. The selective tuning model of attention: Psychophysical evidence for a suppressive annulus around an attended item. Vision Res. 2003;43(2):205–19. doi: 10.1016/s0042-6989(02)00491-1 12536142

56. Müller NG, Kleinschmidt A. The attentional “spotlight’s” penumbra: Center-surround modulation in striate cortex. Neuroreport. 2004;15(6):977–80. doi: 10.1097/00001756-200404290-00009 15076718

57. Müller NG, Mollenhauer M, Rösler A, Kleinschmidt A. The attentional field has a Mexican hat distribution. Vision Res. 2005;45(9):1129–37. doi: 10.1016/j.visres.2004.11.003 15707921

58. Di Russo F, Martinez A, Hillyard SA. Source analysis of event-related cortical activity during visuo-spatial attention. Cereb Cortex. 2003;13(5):486–99. doi: 10.1093/cercor/13.5.486 12679295

59. Martínez A, Anllo-Vento L, Sereno MI, Frank LR, Buxton RB, Dubowitz DJ, et al. Involvement of striate and extrastriate visual cortical areas in spatial attention. Nat Neurosci. 1999;2(4):364–9. doi: 10.1038/7274 10204544

60. Noesselt T, Hillyard SA, Woldorff MG, Schoenfeld A, Hagner T, Jäncke L, et al. Delayed striate cortical activation during spatial attention. Neuron. 2002;35(3):575–87. doi: 10.1016/s0896-6273(02)00781-x 12165478

61. Martínez A, Di Russo F, Anllo-Vento L, Sereno MI, Buxton RB, Hillyard SA. Putting spatial attention on the map: Timing and localization of stimulus selection processes in striate and extrastriate visual areas. Vision Res. 2001;41(10–11):1437–57. doi: 10.1016/s0042-6989(00)00267-4 11322985

62. Mehta AD, Ulbert I, Schroeder CE. Intermodal selective attention in monkeys. I: distribution and timing of effects across visual areas. Cereb Cortex. 2000;10(4):343–358. doi: 10.1093/cercor/10.4.343 10769247

63. Chakravarthi R, Carlson TA, Chaffin J, Turret J, VanRullen R. The temporal evolution of coarse location coding of objects: Evidence for feedback. J Cogn Neurosci. 2014;26(10):2370–84. doi: 10.1162/jocn_a_00644 24738769

64. Zhang Y, Meyers EM, Bichot NP, Serre T, Poggio TA, Desimone R. Object decoding with attention in inferior temporal cortex. Proc Natl Acad Sci. 2011;108(21):8850–5. doi: 10.1073/pnas.1100999108 21555594

65. Noudoost B, Chang MH, Steinmetz NA, Moore T. Top-down control of visual attention. Curr Opin Neurobiol. 2010;20(2):183–90. doi: 10.1016/j.conb.2010.02.003 20303256

66. Mihalas S, Dong Y, von der Heydt R, Niebur E. Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects. Proc Natl Acad Sci. 2011;108(18):7583–8. doi: 10.1073/pnas.1014655108 21502489

67. Wyatte D, Curran T, O’Reilly R. The limits of feedforward vision: recurrent processing promotes robust object recognition when objects are degraded. J Cogn Neurosci. 2012;24(11):2248–61. doi: 10.1162/jocn_a_00282 22905822

68. Muckli L, De Martino F, Vizioli L, Petro LS, Smith FW, Ugurbil K, et al. Contextual Feedback to Superficial Layers of V1. Curr Biol. 2015;25(20):2690–5. doi: 10.1016/j.cub.2015.08.057 26441356

69. O’Reilly RC, Wyatte D, Herd S, Mingus B, Jilk DJ. Recurrent processing during object recognition. Front Psychol. 2013;4(124).

70. Roland PE, Hanazawa A, Undeman C, Eriksson D, Tompa T, Nakamura H, et al. Cortical feedback depolarization waves: A mechanism of top-down influence on early visual areas. Proc Natl Acad Sci. 2006;103(33):12586–91. doi: 10.1073/pnas.0604925103 16891418

71. Roland PE. Six principles of visual cortical dynamics. Front Syst Neurosci. 2010;4:1–21. doi: 10.3389/neuro.06.001.2010

72. Baldauf D, Deubel H. Attentional landscapes in reaching and grasping. Vision Res. 2010;50(11):999–1013. doi: 10.1016/j.visres.2010.02.008 20219518

Feed-forward visual processing suffices for coarse localization but fine-grained localization in an attention-demanding context needs feedback processing

Souhrn

Klíčová slova:

Zdroje

PLOS One

Aktuální možnosti diagnostiky a léčby litiáz