imata logo
Username: Password:  

Forgot Password?
search imata Make a Donation IMATA on Facebook
Masthead Image


Can You Recognize This? Transfer From Objects To Pictures Using A Simple Discrimination Task With California Sea Lions (Zalophus californianus)

By Candyce H. Paparo, Long Island Aquarium and Exhibition Center

Originally published in Soundings Volume 38, Number 3 — Third Quarter 20163

Understanding the cognitive abilities (e.g., object recognition) of marine mammals is an important area of research and one in which researchers, trainers, and the public, are interested. However, studies investigating cognition in marine mammals can be resource intensive and require successful collaboration between facilities (such as aquariums), the animal trainers, and the scientists (see Rodriguez, Guarino, Erb, & Jaakkola, 2012 for a model of how to start up such collaboration at a facility). This type of research is definitely a team effort – whether the team is large or small. We feel that our collaboration, which is between the Long Island Aquarium and Exhibition Center (LIAEC) and their sea lion trainers, and an independent scientist at St. Francis College (SFC), can serve as a model for a research program that is beneficial to the aquarium, the trainers, the researchers, the public, and most importantly – the animals! These types of studies can provide the animals with mental stimulation. Since their participation is voluntary, we must design the procedure to be fun and enriching.

FIGURE 1 - Java decking in front of the pin in the window of the pool enclosure. This was the location of all training and testing for the present study.

We know that many people are interested in having opportunities for collaboration and to be able to conduct research to answer the many captivating and important questions about marine animal cognition. However, the trainers are often in need of expertise for the science aspect of the work while the scientist is in need of the expertise for the training of the animals as well as animal access. Our team from the LIAEC and SFC has been working together for four years, and during this time we have slowly shaped a research program that has successfully fit into the already packed schedule of the animals and trainers. We would like to take this opportunity to describe our study and the research results thus far, and in the process, we hope to inspire other facilities, trainers, and researchers to conduct these types of important cognitive studies with marine animals.

Our project

Our team set out to investigate object to picture recognition in sea lions with two main goals. Our first goal had a practical application, as presently it is common to use pictures displayed on a computer screen as stimuli in a variety of studies on cognition and neuroscience (see Cook, 2001; Laverghetta & Shimizu, 1999). Though a common research method for stimuli presentation, researchers do not know how these two-dimensional images are mentally represented and understood by their research animals. An understanding of this is a critical component to interpreting the results of these types of studies. Previous work investigating the correspondence between real world objects and their two-dimensional images have yielded conflicting results with some demonstrating successful performance supporting an animal’s ability to use pictures to represent objects (Cabe, 1976; Shimizu, 1998; Spetch & Freidman, 2006) while others have not been successful (Dawkins, 1996, Watanabe, 1997). Our study was designed to help clarify this question of whether or not non-humans can use two-dimensional images to represent real world items. Successful performance by our animals adds validity to the cognitive studies that have used, and are currently using, static images as stimuli.

The second, theory-based, goal of our study was to better understand the abilities of non-human animals to represent objects in their environment in a number of ways. This mental skill is critical for survival. For example, an animal must be able to represent a predator in terms of its visual representation, as well as its smell and sound. We wish to further understand the mental abilities that are in place for this to occur, as we believe that a variety of animals likely have the ability to understand a relationship between an object and another representation of that object (e.g., a photograph). While there is a large amount of research on object recognition (Bauer & Philip, 1983; Harley & DeLong, 2008; Herman, 2010; Mendes, Rakoczy & Call, 2011; Pack, Herman, Hoffman-Kuhnt, & Branstetter, 2002) and object discrimination (Autier-Dérian, Deputte, Chalvet-Monfray, Coulon, & Mounier, 2013; Mauck & Dehnhardt, 2005; Yuki, Mika, Hidetoshi, & Akira, 2012) in a variety of mammal species, research on object to picture recognition has primarily involved birds as subjects, such as pigeons and chickens. There has been much debate in the literature over whether non-humans can understand the correlation between pictures of objects and the actual objects the pictures represent. As mentioned above, these types of studies have yielded conflicting results (see Fagot, Martin-Malivel, & Depy, 1999). Therefore, it has been suggested that additional research on object-picture recognition in various non-human mammals is needed (Bovet & Vauclair, 2000). Our present study adds to this research area by investigating whether or not sea lions can use two-dimensional photographs to represent three-dimensional objects using a Simple Discrimination Procedure. In this procedure, a subject is presented with two different objects and rewarded for choosing the correct choice of the two objects (which was pre-determined by the researcher).

Broadly speaking, our project provides further insight into how animals represent their world and adds to our general knowledge on marine mammal cognition, which is of large interest to both the scientific community and to the public. For example, as mentioned above, one of the final results (this is currently a work in progress) will provide either support or opposition for using pictures as stimuli in cognitive studies with non-humans - a process which is already occurring in many research programs. Another exciting aspect of this work is that it is currently being used as an educational tool. Undergraduate research assistants at SFC are able to learn about, and participate in, the research design, data collection, and data analysis components of the research.  Also, at the LIAEC, sessions are run in view of the public for their observation followed by a discussion with the trainers, and of course, these sessions provide enrichment, in the form of mental stimulation, for the sea lions.

FIGURE 2 - Stimuli used for training and testing. The pin (A) was the correct choice and the ball (B) was the incorrect choice as determined by the researcher. Training stimuli were the actual objects and the testing stimuli were two-dimensional, laminated, color photographs of the training objects printed to scale.

Training and procedure

We began this study with a number of challenges ahead of us. The first challenge was to train a simple discrimination procedure with our sea lions – Bunker (female) and Java (male). Both animals were 11 years old at the time of the experiment and had previous extensive training for husbandry and aquarium shows/demonstrations but neither had participated in a behavioral research study prior to the current work. Training sessions were run poolside through the opening of the window of the enclosure’s surrounding wall. The sea lions already knew how to perform a behavior called decking in which they placed their chin on the deck of this open window ledge. Since that was where the experimental apparatus would eventually be placed we decided to begin training by using the decking behavior as the sea lion’s response to indicate their stimulus choice (Figure 1). Only one animal was trained in the enclosure at a time and stimulus presentation was counterbalanced so that the two stimuli (a toy ball and toy bowling pin; (Figure 2) were presented equally on the left and right sides. To begin a training session the trainer called the sea lion up to the open window of the enclosure, tossed a fish to the far end of the pool, and while the animal was swimming the trainer would place the two stimuli on the ledge. When the sea lion returned they would be reinforced for decking in front of the researcher-chosen correct stimulus (which was the toy bowling pin for both animals). They were initially prompted with a visual cue by the trainer to deck in front of the pin, as they had never had to deck without the explicit signal (a finger pointed down) from the trainer. After four days that prompt was faded out and the animals approached the stimuli and decked without any signal from the trainer.

Once Bunker and Java were decking, without a cue from the trainer, in front of the correct stimulus, the next phase of training began. This phase consisted of training the sea lions to interact with the apparatus by placing it into the window of the habitat (Figure 3). This gave the animals access to the apparatus and the stimuli for sessions when in the water. The trainers were positioned on land outside the enclosure (behind the apparatus) and observed the animal’s behavior with a mounted mirror. The apparatus allowed for necessary experimental controls but also presented a number of training challenges. First, the sea lions had to be desensitized to the apparatus, which luckily did not take long. This process consisted of one session a day per animal, four days a week, for two weeks.  Bunker and Java were provided with positive reinforcement for approaching the apparatus on land and in the water (with the apparatus in the window) and having the apparatus in place in the window during regular training sessions that were unrelated to the apparatus itself. Once the sea lions were desensitized we began to train them to position on a T-bar chin station in front of the apparatus. This positioned the animals 0.4 meters (1.31 feet) in front of the apparatus and centered them equally between the stimulus boxes. Next the sea lions were trained to a release tone, which served as a cue for them to break from holding on the station and approach the apparatus to make their choice between the objects presented. This allowed for experimenter/trainer control over the length of time that Bunker and Java could view the stimuli prior to making their choice and also provided a way to communicate to the animals that they should make a choice for future studies in which novel stimuli would appear.

Throughout the entire training process there were two ongoing challenges that the trainers worked on. These were to increase the number of trials per day that the sea lions were running and to decrease and make uniform, the number of fish reinforcers used per correct trial. At the beginning of our project the animals were running an average of one five-trial session per day, three days per week. One major reason for this was that we were simply trying to accommodate research training into an already packed schedule for both the trainers and the animals. We figured that ‘slow and steady (and realistic goals) would win the race’. Eventually we were able to successfully accommodate research into the weekly schedule and were routinely running each sea lion on one 20-trial session three to four days per week. It took us approximately two years to sort out all of the logistics and to reach this long-term goal.

Another reason for the initial low number of trials per day was that the animals were receiving variable reinforcement and jackpots during preliminary research training to keep their initial motivation high and frustration levels low. This put a large constraint on the number of behaviors/trials that could be worked on each session/day. Over time, as the animals’ intrinsic motivation for the task increased, we were able to substantially decrease the fish required for general training. Also, for experimental controls, we were able to make the number of fish for each correct trial uniform with Bunker and Java each receiving one fish per correct response.

At this point, after approximately three years, the general procedure and experimental controls were in place. The animals were trained to interact with the apparatus with the trainers out of view. During sessions, two trainers were behind the apparatus. Trainer One controlled the presentation of the stimuli and the feedback given to the sea lion (correct versus incorrect responses). Trainer Two assisted by nonverbally informing Trainer One of which stimuli to place in the appropriate stimulus boxes as well as recording the responses of the animal. The stimuli in the boxes were always concealed from the animal by sliding doors until the beginning of each trial. The stimuli were always placed simultaneously into the boxes so that the animal could not inadvertently be cued by either sound or motion of the trainer behind the apparatus. All sessions were video-recorded and the animals’ responses were later confirmed by another reviewer.

A trial began when the trainer opened the stimulus box doors, simultaneously revealing the ball and the pin to the animal. The sea lions were trained to hold on the station from zero to four seconds, but generally the release tone was given after approximately one second, allowing the animal time to settle on the station. A response by the sea lion was defined as moving from the center station to deck on a stimulus box, and to hold the response for one second. Correct responses were signaled to the animal by a one-second whistle (ACME high-pitch, non-adjustable whistle) that already served as a conditioned reinforcer and was followed immediately by one piece of fish tossed directly over the center of the apparatus from behind. The comparison doors were then simultaneously closed, signaling the end of the trial. Incorrect responses were not reinforced and instead were followed by the simultaneous closing of the comparison doors at the end of each trial.

FIGURE 3 - Bunker in front of the apparatus used for training and testing. The photograph on the top left (A) is Bunker stationed prior to the stimulus doors opening. The photograph on the top right (B) is Bunker making her stimulus choice by decking in front of the box containing the pin. The bottom photograph (C) is Bunker making her stimulus choice by decking in front of the box containing the pictorial stimulus of the pin during the test session.

The test and results

VIDEO 1: Java Training Video

VIDEO 2: Java Training Video

After approximately two months of running one 20-trial session per day, three days a week, with the apparatus as described above, both Bunker and Java performed at a level of 90% correct responses or higher on four consecutive sessions. We were then ready to run our single Transfer Test. Each sea lion participated in one test session which consisted of the same general methods from above but rather than using the three-dimensional objects of the ball and pin as the stimulus choices the sea lions were exposed to two-dimensional, laminated color photographs of the objects that were made to scale. Each sea lion began the test session with a warm-up consisting of four trials with the physical objects in order to assess motivation and baseline performance. On the 5th through 25th trial the apparatus doors opened to reveal the photographic stimuli rather than the objects themselves. Our main research question was ‘What will Bunker and Java do?’ Will they respond to the novel pictures in the same way as they had to the trained objects? This would indicate a transfer of knowledge from what they were explicitly trained on regarding the objects to the photographs.

Both Bunker and Java answered this question with a resounding yes (Figure 3). They each responded correctly on their first exposure to the photographs. Overall, on her test session Bunker had an 80% success rate and Java had a 90% success rate. Thus, each sea lion performed significantly above chance levels (p<0.05; Two-Tailed Binomial Test) and were not significantly different from their baseline levels (p>0.05 Fischer’s Exact Test), indicating that they were responding to the objects and the pictures of the objects in generally the same manner. Behaviorally they showed some hesitation to approach the novel stimuli (i.e., the photographs) in the beginning of the session, indicating that they were aware of a difference in the stimuli presented yet they still made the correct choice on this discrimination task. However, as exciting as these results are, it is important to note that the data represents the first piece of a larger puzzle. There are further experiments that need to be conducted in order to more confidently state that these animals can use pictures to represent objects in their environment. The current work presented here is a first step. We plan on replicating this study using different stimuli in order to control for variables such as the shape and color of the objects. We also plan to investigate other aspects of object recognition and will continue this productive collaboration between the LIAEC and SFC.

The effort has paid off!

We now have preliminary evidence of the ability of two sea lions to use photographs to represent objects.  This adds support for the practical application and use of computer images as stimuli for animals in various research studies, and also supports the theoretical reasoning that non-human animals have the mental ability to represent objects in their environment in a variety of ways.

The knowledge that we have gained from this study has been the culmination of over four years of work. The trainers at the LIAEC have worked incredibly hard in order to create and sustain a successful research program with their two resident California sea lions. The researcher has had to work from a distance, and also design, and occasionally redesign, the methods in order to accommodate the LIAEC training schedule/protocol as well as individual animal personalities. As trainers and researchers we have all found this to be incredibly rewarding and we are looking forward to the future of our work together. We hope that our discussion of the pace and process by which we began this exciting, important, and enriching work will motivate others and serve as an example of how research collaboration between an aquarium and an independent scientist can be a realistic goal and a fruitful experience – it just takes some time to find a balance for all parties. Think big but start small. It is amazing what we can do together!

Authors’ Note

We would like to acknowledge all of the staff at the Long Island Aquarium and Exhibition Center, in particular: Erika Culmo, Anne Haas, Valerie Ruggeri Olstad, and the Director of Animal Training, Ann Yaiullo.  We would also like to acknowledge the SFC student research assistants: Alexis Acevedo, Jeannette Raymond, and Brian Restrepo. Without all of their hard work and dedication this research could not have been carried out.

This research was approved by the Ethics Review Board at St. Francis College and partially supported by the St. Francis College Faculty Research Grant.


Autier-Dérian, D., Deputte, B. L., Chalvet-Monfray, K., Coulon, M., & Mounier, L. (2013). Visual discrimination of species in dogs (Canis familiaris). Animal Cognition, doi:10.1007/s10071-013-0600-8

Bauer, H. R., & Philip, M. (1983).  Facial and vocal recognition in the common chimpanzee.  The Psychological Record, 33, 161-170. Retrieved from

Bovet, D., & Vauclair, J. (2000). Picture recognition in animals and humans. Behavior and Brain Research, 109, 143-165. doi:10.1016/S0166-4328(00)00146-7.

Cabe, P. A. (1976). Transfer of discrimination from solid objects to pictures by pigeons: A test of theoretical models of pictorial perception. Perception & Psychophysics, 19(6), 545-550. doi:10.3758/BF03211224.

Cook, R. G. (2001). Avian Visual Cognition. Retrieved May 28, 2013 from

Dawkins, M. S. (1996). Distance and social recognition in hens: Implications for the use of photographs as social stimuli. Behavior, 133, 663-680. doi:10.1163/156853996X00413.

Fagot, J., Martin-Malivel, J., & Depy, D. (1999). What is the evidence for an equivalence between objects and pictures in birds and nonhuman primates? Cahiers De Psychologie Cognitive/Current Psychology of Cognition, 18, (5-6), 923-949.

Harley, H. E., & DeLong, C. M. (2008). Echoic Object Recognition by the Bottlenose Dolphin. Comparative Cognition & Behavior Reviews, 346-65. doi:10.3819/ccbr.2008.30003

Herman, L. M. (2010). What laboratory research has told us about dolphin cognition. International Journal of Comparative Psychology, 23(3), 310-330.

Laverghetta, A. V., & Shimizu, T. (1999). Visual discrimination in the pigeon (Columba livia): Effects of selective lesions of the nucleus rotundus. Neuroreport: For Rapid Communication of Neuroscience Research, 10(5), 981-985. doi:10.1097/00001756-199904060-00016

Mauck, B., & Dehnhardt, G. (2005). Identity concept formation during visual multiple-choice matching in a harbor seal (Phoca vitulina). Learning & Behavior, 33(4), 428-436. doi:10.3758/BF03193181Shimizu, T. (1998). Conspecific recognition in pigeons (Columbia livia) using dynamic video images. Behavior, 135, 43-53.

Mendes, N., Rakoczy, H., & Call, J. (2011). Primates do not spontaneously use shape properties for object individuation: A competence or a performance problem?. Animal Cognition, 14(3), 407-414. doi:10.1007/s10071-010-0375-0

Pack, A. A., Herman, L. M., Hoffmann-Kuhnt, M., & Branstetter, B. K. (2002). The object behind the echo: dolphins (Tursiops truncatus) perceive object shape globally through echolocation. Behavioural Processes, 58(1/2), 1.

Rodriguez, M. L., Guarino, L.S., Erb, S.L., & Jaakkola, K.O. (2012). Studying the minds in the water: Practical tips for conducting cognitive research with marine mammals. Soundings, 37(3), 11-13.

Spetch, M. L., & Friedman, A. (2006). Pigeons see correspondence between objects and their pictures. Psychological Science, 17 (11), 966-972. doi:10.1111/j.1467-9280.2006.01814.x.

Watanabe, S. (1997). Visual discrimination of real objects and pictures in pigeons. Animal Learning and Behavior, 25, 185-192. doi:10.3758/BF03199057.

Yuki, H., Mika, S., Hidetoshi, K., & Akira, M. (2012). Visual discrimination in an orangutan (Pongo pygmaeus): measuring visual preference. Perceptual & Motor Skills, 114(2), 429-432. doi:10.2466/22.24.PMS.114.2.429-432.