The Risk of Building Emotional Ties with Responsive AI

Seidenberg School of CSIS

Curious to learn what my colleagues at the Seidenberg School of Computer Science and Information Systems at Pace University thought, I asked some faculty members for their thoughts on how anthropomorphization would impact users from a psychological perspective.

A man in a suit shaking hands with a cybernetic arm.
A man in a suit shaking hands with a cybernetic arm.
Katie Todd

In an August safety report outlining some of the risks identified in the newly released GPT-4o, OpenAI conducted research on the possibilities of users forming an emotional reliance on the generative AI platform. Through a psychological phenomenon called anthropomorphization, users could attribute human-like qualities to the chatbot and begin to liken it to a real person–which raises many red flags regarding the impact on that user's ability not just to think critically about the information they receive from GPT-4o, but how they engage with other people in their lives.

As someone in a human-centered profession, the thought of people depending on algorithmic means of emotional fulfillment doesn't exactly get me excited. Relationships are at the core of human contentment, and AI cannot provide a relationship that is authentically reciprocal. It can appear that way, which is problematic. AI itself is not the issue. It's more what is missing in society that might cause people to turn to artificial intelligence to have their needs met. Increased siloing of individuals following the pandemic, phone addiction, social media's influence in polarization of thought and opinion, and a lack of third spaces for people to get together and make new connections. GenAI is a tool, not a replacement for real social connections.

Curious to learn what my colleagues at the Seidenberg School of Computer Science and Information Systems at Pace University thought, I asked some faculty members for their thoughts on how anthropomorphization would impact users from a psychological perspective. See below for some responses from not just AI experts but also professors specializing in Human-Centered Design.

Dr. Juan Shan, Associate Professor of Computer Science

The voice model is amazing from the technical perspective. It is another surprise and milestone from OpenAI. At the same time, I hear people’s worries about the possible negative sides, such as becoming emotionally hooked, ethical issues, security issues, and so on. I share those worries too. If we look back at tech history, there are always positive and negative sides to new technologies, and there are always controversies about new technological advancements. In my opinion, what we can do is to keep ourselves knowledgeable about advancements, use them, test them, and help shape them. We also need to educate our students for both technical skills and strong ethical standards. At the same time, the government should take more responsibility to investigate and lead the development of AI, to detect possible misuse of AI, to estimate possible consequences, and to regulate the publication process of AI products. My optimal vision for the future is that people can enjoy the benefits and convenience brought by AI in daily life, while potential harm is under control and made known to the public.

Dr. Jonathan Williams, Clinical Assistant Professor, Human-Centered Design

The companionship and emotional lives of objects and tools has long been established, but this is a new era where the tool mirrors that relationship back to the user. Humans have the capacity to emote and attach to technology, but the joy, hope, or love they may receive back through AI will be algorithmically defined. While there may be emotional ties from a human to the AI, authentic reciprocity is not received in return.

Human to human emotion takes on a full spectrum of behaviors, feelings, and thoughts. Generative AI is heavily moderated and censored. Generative AI can't get angry or profane, mourn or grieve, or call on personal experiences. For the many emotions we may offer to generative AI, only a select few can be returned to us.

Dr. Zhan Zhang, Associate Professor, Director of Human-Centered Design

People forming emotional ties with AI tools is not a new phenomenon; it has been observed in prior research involving human interactions with voice assistants like Alexa and social robots. Emotional connections with AI are a nuanced and complex subject. On one hand, AI can offer significant emotional support, such as companionship or empathy, by engaging users in meaningful conversations, particularly when it is designed to mimic human-like voices. However, these emotional ties raise critical questions and concerns. For example, forming emotional connections with AI often involves sharing personal thoughts and feelings, which brings up issues of data privacy and the potential misuse of this sensitive information. Moreover, since AI does not possess genuine emotions, any emotional connection felt by the user is inherently one-sided. This creates ethical concerns about the possibility of manipulating users' emotions. More research is needed to investigate this fascinating topic, which has significant societal implications. In particular, human-computer interaction (HCI) researchers can play a crucial role in examining emotional ties with GenAI from both sociotechnical and ethical perspectives.

Contains Video
No

More from Pace

Gabriel Pell and Fernando Mendoza Lopez Join the Elisabeth Haub School of Law at Pace University as Visiting Assistant Professors

Elisabeth Haub School of Law

The Elisabeth Haub School of Law at Pace University is pleased to announce that Gabriel Pell and Fernando Mendoza Lopez have joined its faculty as Visiting Assistant Professors of Law. Professor Pell is a former prosecutor who comes to Haub Law after having taught for several years at New York Law School. Professor Mendoza Lopez recently received his JD and PhD from Vanderbilt Law School, and previously served as Director of Economic Studies in the Mexican federal government.

Elisabeth Haub School of Law at Pace University Visiting Assistant Professors Mendoza Lopez and Pell
Image
Elisabeth Haub School of Law at Pace University Visiting Assistant Professors Mendoza Lopez and Pell

The Elisabeth Haub School of Law at Pace University is pleased to announce that Gabriel Pell and Fernando Mendoza Lopez have joined its faculty as Visiting Assistant Professors of Law. Professor Pell is a former prosecutor who comes to Haub Law after having taught for several years at New York Law School. Professor Mendoza Lopez recently received his JD and PhD from Vanderbilt Law School, and previously served as Director of Economic Studies in the Mexican federal government.

"The Elisabeth Haub School of Law at Pace University prides itself on bringing unique teaching perspectives to the classroom,” said Elisabeth Haub School of Law at Pace University Dean Horace E. Anderson Jr. “Our Visiting Professors are both scholars and practitioners whose diverse experience will benefit the Haub Law student experience. We are thrilled to welcome Professors Pell and Mendoza Lopez to Haub Law.”

Professor Pell's research and writing interests focus on criminal law and criminal procedure. He speaks and writes about the application of traditional search-and-seizure practices and other investigatory techniques to new technologies in the digital era. Before joining the Elisabeth Haub School of Law at Pace University as a Visiting Assistant Professor of Law, Professor Pell served as an Assistant Professor of Law at New York Law School. Prior to entering academia, Professor Pell spent the bulk of his practicing career as a prosecutor, most recently as an Assistant District Attorney specializing in appeals and post-conviction matters with the Middlesex District Attorney’s Office in Massachusetts. Professor Pell also served as an Assistant District Attorney with the Suffolk County District Attorney’s Office and has held positions in the private sector as well.

Following his graduation from law school, Professor Pell clerked for the Honorable F. Dennis Saylor IV of the United States District Court for the District of Massachusetts. Later in his career, Professor Pell clerked for the Honorable Gary S. Katzmann at the Massachusetts Appeals Court. He earned his JD from Harvard Law School and his AB from Princeton University.

“Haub Law has a strong reputation in criminal law – both in its graduating students who go on to become successful Assistant District Attorneys and in its faculty who are experts in the field,” said Professor Pell. “I look forward to sharing my research and experience in this field with the Haub Law community and collaborating with its faculty and students.”

Professor Mendoza Lopez’s research interests include the intersection of public policy and government contracting, with a focus on investigating how procurement processes can be leveraged to achieve broader societal objectives. Prior to entering graduate school, Professor Mendoza Lopez served as Director of Economic Studies in the Mexican federal government, where he oversaw procurement strategies for a range of public goods and services. His experience includes developing policies to enhance competition, promote small businesses, and incorporate environmental considerations into purchasing decisions.

Professor Mendoza Lopez holds a BA and MA in Economics, from the Tecnologico de Monterrey, and the University of Texas at Austin, respectively. In 2024, he earned his JD from Vanderbilt Law School and his PhD in Law and Economics from Vanderbilt University. Additionally, he holds a Masters in International Trade and Finance from Yonsei University’s Graduate School of International Studies in South Korea.

“Throughout my career I have focused on developing positive policies to better society and I cannot think of a better institution to not only share my research towards this end, but to learn from an institution who has been doing exactly that since its founding,” said Professor Mendoza Lopez. “Haub Law’s many centers and clinics are devoted to using the law to better society and I am thrilled to have an opportunity to be part of that end goal.”

In 2023, the Elisabeth Haub School of Law at Pace University appointed Paul Rink and Salihah Denman as Visiting Assistant Professors. Professors Rink and Denman will each remain at Haub Law for a second year as Visiting Assistant Professors for the 2024–2025 academic year.

Contains Video
No

More from Pace

Press Release: Mandy Moore Named Director of Pace University’s Commercial Dance LA Semester

Sands College of Performing Arts

Pace University’s Sands College of Performing Arts has named Mandy Moore as director of its Commercial Dance LA semester. The BFA Commercial Dance Program was founded in 2012 and launched its first full semester in LA in 2018. The Commercial Dance program brings 35 students to Los Angeles in the spring semester of their junior year. The renowned choreographer will teach the choreography for its camera course and direct the showcase course.

Mandy Moore, Pace University's Director of Commercial Dance LA
Mandy Moore, Pace University's Director of Commercial Dance LA

Renowned choreographer to lead and innovate prestigious dance program’s semester in LA

Pace University’s Sands College of Performing Arts has named Mandy Moore as director of its Commercial Dance LA semester. The BFA Commercial Dance Program was founded in 2012 and launched its first full semester in LA in 2018. The Commercial Dance program brings 35 students to Los Angeles in the spring semester of their junior year. The renowned choreographer will teach the choreography for its camera course and direct the showcase course.

Moore’s exceptional career is marked by groundbreaking achievements and a profound impact on the dance and entertainment industry. With three Emmy Awards among her 13 nominations, she has captivated audiences worldwide through her work on iconic productions such as "So You Think You Can Dance."

Moore made history in 2017 as the first choreographer to work on the Golden Globe Awards, Academy Awards, Grammy Awards, and Emmy Awards in the same year. Some of her most recent work includes Paramount’s “Babylon” and “Taylor Swift’s “The Eras Tour.”

Her choreographic artistry has been showcased in acclaimed films such as Chazelle’s original "La La Land," and "American Hustle," as well as in television shows like "Zoey's Extraordinary Playlist," and "Glee." Her stage work includes directing and choreographing the “Dancing with the Stars Tour,” “The Wedding Singer,” and “Nobody Loves You.”

Jennifer Holmes, who was recently named inaugural dean of the Sands, is pleased to have Moore as the program’s new LA semester director. “It is a tremendous honor to have Mandy Moore as the director of our Commercial Dance LA Semester,” said Holmes. “Her numerous awards and achievements as well as her limitless talent for teaching choreography is the ultimate compliment to our program. Our students are in the fortunate position of being able to learn from the best in the business.”

Moore is joining the Sands College of Performing Arts at an exciting time. The program is selective and only takes 30-35 students each year out of approximately 1,000 applicants. In fact, for the second year in a row, Pace has been named one of the top ten most represented colleges on the Broadway stage by Playbill, one of the leading theatre news publications.

“I am looking forward to getting in the classroom and experiencing all the wonderful light bulb moments that happen in the learning process,” said Moore. “I am also honored to be a resource for each one of these dancers as they travel along their artistic path.”

The appointment comes at an exciting time for the program. The Sands College of Performing Arts recently celebrated the graduation of its inaugural class in May. This milestone comes after the program became the seventh school and college within Pace, thanks to a $25 million gift from Pamela and Rob Sands, J.D. ’84.

The Sands donation is part of a larger campaign for the more than $200 million transformation of One Pace Plaza, its flagship building in lower Manhattan. This transformation will include new academic spaces, a modernized residence hall, and a new state-of-the-art Performing Arts Center. The Performing Arts Center will include a 450-seat proscenium theater, a 200-seat flexible theater, and a 99-seat black box theater. The new space will also feature scene and costume shops, dressing rooms, rehearsal rooms, green rooms, dance studios, and public gathering spaces.

About Pace University

Since 1906, Pace University has been transforming the lives of its diverse students—academically, professionally, and socioeconomically. With campuses in New York City and Westchester County, Pace offers bachelor, master, and doctoral degree programs to 13,600 students in its College of Health Professions, Dyson College of Arts and Sciences, Elisabeth Haub School of Law, Lubin School of Business, Sands College of Performing Arts, School of Education, and Seidenberg School of Computer Science and Information Systems.

About Sands College of Performing Arts

Sands College of Performing Arts is a cutting-edge performing arts school that nurtures artistic innovation and develops future leaders and changemakers in the profession. Renowned for its diverse range of highly-ranked programs, Sands College is committed to cultivating top-talent performers, designers, and production professionals. From Hollywood to Broadway, Sands College students and alumni have appeared in major film and television productions and theater performances such as & Juliet, A Beautiful Noise, A Strange Loop, Almost Famous, Hadestown, Hamilton, Kimberly Akimbo, MJ The Musical, Moulin Rouge, Ohio State Murders, Wicked, and the 2022 Radio City Christmas Spectacular. Recognized by Playbill in 2023 as one of the 10 most represented colleges on Broadway, Sands College continues to inspire the next generation of artists on stage and screen.

Contains Video
No
In The Media

More Press Releases

Giving Voice to Social Justice: Pace Students Partner with the Bread and Puppet Theater

Dyson College of Arts and Science

In spring 2024, students from three courses, The Artist as Activist, Nonviolence: Theory and Practice, and Dramaturgy 1 engaged with the world-renowned, Vermont-based social justice theater company, Bread and Puppet, for an afternoon workshop at the Judson Memorial Church in New York City.

Pace University's Peace and Justice Studies and English students sitting in a circle and working with the Bread and Puppet Theater to express social justice through art.
Antonia Gentile

In spring 2024, students from three courses, The Artist as Activist, Nonviolence: Theory and Practice, and Dramaturgy 1 engaged with the world-renowned, Vermont-based social justice theater company, Bread and Puppet, for an afternoon workshop at the Judson Memorial Church in New York City.

In small groups, and guided by a Bread and Puppet puppeteer, they wrote and performed mini cantastoria (from the Italian “canta storia,” or “sung story”), a form of street theater using song, movement, text, and puppetry that is championed by Bread and Puppet.

Image
Pace University's Peace and Justice Studies and English students sitting in a circle and working with the Bread and Puppet Theater to express social justice through art.
Photo by Asya Gorovits

Timely themes such as student loan debt, mass incarceration, and the destruction of the planet were given voice and brought to life by musical instruments students learned to play, such as cymbals, bass drums, and tambourines, as well as artistic backdrops created on large sheets of white paper.

After the conclusion of the workshop, Bread and Puppet performed for a full house at Judson. Adjunct Professor Aaron Ellis, PhD, Sands College, organized and served as facilitator of a post-show discussion. It featured three members of the Bread and Puppet Theater, Professor of English Sid Ray, PhD, and Associate Professor and Chair of Peace and Justice Studies Emily Welty, PhD, faculty who, in addition to Ellis, mentored students throughout this unique opportunity.

The arts meet activism

Ellis, an activist-in-residence for the Peace and Justice Studies program in spring 2024, was a key factor in bringing the Bread and Puppet theater to Pace students, working with their colleagues to envision, plan, and facilitate the workshop as well as the post-show discussion by Ray and Welty, the subject of the latter including the overlap and role of radical arts and social justice teaching today.

It was an afternoon filled with interdisciplinary learning, growth, and collaboration. And according to Ellis, it was through this opportunity that students deeply engaged in exercising the basic muscle necessary for theater: using art, collaboration, and teamwork to address issues of great importance for our times.

“Participants–even those with theater training–learned new performance traditions, skills, and ways of demonstrating their activism and advocacy through performance, in community. They learned these new skills under the strong facilitation and guidance from experts in the field and tried and succeeded in creating new things, supported by, and supporting their peers,” they said.

Aja Ammeen ’26, Peace and Justice Studies, was one of the students enrolled in this workshop. “I learned how to incorporate other forms of art into activism and how to share it with others without over complicating the performance or message. Overall, the experience was wonderful, and it's something I love to reminisce about,” she said.

Funding for experiential learning at Pace

An example of experiential learning at Pace that builds on classroom experience yet is both outside a traditional civic engagement course and involves faculty engagement, this workshop was one of nine projects funded to date by Dyson’s Experiential Learning Innovation Fund.

The goal of the fund is to advance efforts to build the Pace community internally and differentiate it as a leader in experiential education. And Kelley Kreitz, PhD, director of experiential learning and the Pace Path, is pleased to witness the fund’s mission come to fruition through this opportunity.

She said, “Experiential learning at Pace empowers students to learn in and with our communities in New York City and the Lower Hudson Valley and to become makers of the world in which they want to live.”

Contains Video
No
Related Profiles

Sharpening Digital Storytelling Skills

Lubin School of Business

Lubin student Bianca Mazur '27 is interning and sharpening her skills in social media, publicity, community engagement, analytics, influencer relations, and brand management.

Pace University Lubin School of Business student Bianca Mazur '27

Bianca Mazur

Class of 2027

Currently Studying: BBA in Advertising and Integrated Marketing Communications and Sports Marketing (double major) / MS in Social Media Mobile Marketing (combined degree) with a minor in Journalism and Digital Storytelling

Current Internship: Marketing Intern, Warner Bros. Studio Tour Hollywood

Pronouns: She/Her

Image
Pace University Lubin School of Business student Bianca Mazur '27

Lubin student Bianca Mazur '27 is interning and sharpening her skills in social media, publicity, community engagement, analytics, influencer relations, and brand management.

How has the Lubin School of Business helped you prepare for your current internship?

The Lubin School of Business has helped me prepare for my current internship by giving me a strong foundation for professional experiences. My business core courses have provided me with a great understanding of the principles of marketing and management, which I use every day in my internship. As Digital Marketing Coordinator for the American Marketing Association, I ran the organization's social media accounts, which served as an amazing chance to create content and handle community management, two of the tasks I work on every week at Warner Bros. Discovery (WBD). Additionally, completing the ASPIRE Program through Career Services has helped me perfect my response to "tell me about yourself," which has been asked in the 30+ informational interviews I've had so far this summer.

What attracted you to this internship?

I interned with Warner Bros. Discovery last summer and had a blast, so I was more than happy to return. I knew that working with the marketing team for the Studio Tour would expose me to a variety of aspects of marketing, including social media, publicity, community engagement, analytics, influencer relations, and brand management. I was eager to gain experience in the different areas of marketing, so this internship is an excellent fit for me. Furthermore, I really enjoyed being part of WBD's internship program last year, and I was excited to come back to meet all the new interns and have another great summer with the company.

Describe what a typical day looks like as an intern at your organization?

Since my department comes into the office every day, my morning starts at my desk at 8:30 a.m. After saying "hi" to the team, I respond to emails and then open Sprout Social to do some community management for our social media accounts. All of my department meetings are before lunch because we work with some people on the east coast. We discuss upcoming marketing campaigns, social media strategy, or data analytics, depending on the day. The rest of my morning consists of coffee chats, informational interviews, and intern events. I try to schedule at least three meetings per week with someone outside of the Studio Tour department to learn about their role for WBD, since there are so many cool people across the company's brands. I'm able to meet lots of interns in Los Angeles through the frequent events hosted by Early Talent, whether it's a networking mixer or a speaker panel with executives. After getting lunch with my team at the Commissary, I work on my projects for the week. They vary based on what's going on, which keeps things fresh and interesting. Sometimes I'll conduct market research, write copy for our newsletter, or head down to the Studio Store to take pictures of new products. As the day winds down, I send out some emails to schedule informational interviews, and I'm done at 5:00 p.m.

What has been the highlight or most enjoyable aspect of your internship so far?

The highlight of my internship has definitely been the people. As part of the Reach Honorship Program, this is my second summer interning with WBD, and I've loved catching up with everyone I met last summer. It's great to be back at the company and see the other Reach interns, my former team and mentors, and others I previously connected with. Moreover, my team this year is wonderful; they get me involved in projects and opportunities to help me build my marketing skills and accomplish my goals for the summer. At the beginning of my internship, I sat down with my manager and explained what I wanted to get out of this experience, and the team has made sure that I'm able to meet these goals, which I'm incredibly grateful for.

How has this experience shaped or impacted your career goals?

This experience has helped me realize how much I enjoy creating content for social media, which makes me thrilled to continue down this path with my combined degree track, where I will be getting a MS in Social Media and Mobile Marketing. I love coming up with ideas for posts, writing scripts and outlines, filming, and editing everything together to make a great final product. I really feel that my majors, minors, and master's degree are the perfect combination for my interests. This internship has solidified my career goals and given me the opportunity to meet people willing to help me achieve them.

How will this experience impact your next semester at Lubin?

This experience has helped me develop my marketing skills, which I will put to use in campaigns and projects for my Special Events Marketing for Arts, Entertainment and Sports course this fall semester. Additionally, I'm excited to apply what I've learned this summer to my leadership positions at Pace. Creating content that aligns with a brand's values this summer has given me the opportunity to strengthen my social media management and digital storytelling skills, which will come in handy as the Social Media Marketing Manager for POPTV, Pace's multimedia and pop culture club. I also hold the position of Chief Operations Officer for the American Marketing Association, which requires project management, team collaboration, and effective communication skills. I've been able to grow all of these by working on multiple marketing projects each week this summer, so I'm really looking forward to seeing what my next semester at Lubin will bring.

As Wayne Gretzky said, "you miss 100% of the shots you don't take." You never know how one action could impact your future unless you go for it, so send that email, make that connection, and keep in touch. Networking is so important at an internship, so make the most of your time there by finding people who can support you and help you achieve your goals.

Do you have any advice for other Lubin students who would like to pursue an internship in the future?

As Wayne Gretzky said, "you miss 100% of the shots you don't take." You never know how one action could impact your future unless you go for it, so send that email, make that connection, and keep in touch. Networking is so important at an internship, so make the most of your time there by finding people who can support you and help you achieve your goals. Also, Career Services will be your best friend when applying for an internship. They're great to visit for resume help, interview prep, and more. I highly recommend them!

Connect with Bianca:

Contains Video
No

Gaining Insight in the Field of Finance and Tax

Lubin School of Business

Lubin student Emiliah Barcesa '25 is building relationships during her internship, opening doors to valuable opportunities, and gaining insight in the field of finance and tax.

Pace University Lubin School of Business student Emiliah Barcesa '25

Emiliah Eisen L. Barcesa

Class of 2025

Currently Studying: BBA in Public Accounting

Current Internship: Corporate Tax Intern, MetLife

Pronouns: She/Her

Image
Pace University Lubin School of Business student Emiliah Barcesa '25

Lubin student Emiliah Barcesa '25 is building relationships during her internship, opening doors to valuable opportunities, and gaining insight in the field of finance and tax.

How has the Lubin School of Business helped you prepare for your current internship?

The Lubin School of Business has provided me with a strong foundation that complements my current internship at MetLife. It has helped me develop a practical understanding of financial principles, taxation, and business analytics, which are essential for navigating real-world tasks and challenges in a corporate environment.

The Lubin School of Business has provided me with a strong foundation that complements my current internship at MetLife. It has helped me develop a practical understanding of financial principles, taxation, and business analytics, which are essential for navigating real-world tasks and challenges in a corporate environment.

What attracted you to this internship?

As a previous intern at Deloitte, I had the opportunity to explore their largest clients, and MetLife particularly caught my attention due to its prominence as one of the largest insurance companies. My interest in tax was further sparked by understanding the complex financial operations of such large corporations. This motivated me to delve deeper into tax-related matters during my internship experiences.

Describe what a typical day looks like as an intern at your organization?

As an intern at my organization, my typical day involves a dynamic mix of learning and hands-on experience in finance and tax. I attend meetings in the morning, where we discuss ongoing projects, such as transfer pricing and restructuring. Then, throughout the day, I work on assigned tasks such as analyzing market trends and preparing documentation for the Global Tax department. I recently was also assigned a research project, particularly focusing on FIN 48 (Accounting for Uncertainty in Income Taxes).

What has been the highlight or most enjoyable aspect of your internship so far?

The networking and collaboration aspects of my internship have been particularly enjoyable. Meeting professionals from diverse backgrounds and interacting with fellow interns during events provides valuable insights and fosters a supportive learning environment.

How has this experience shaped or impacted your career goals?

Engaging with professionals with different perspectives and collaborating with fellow interns has influenced my career goals. The internship provides insights into various career paths within the industry, expands professional networks, and offers opportunities to learn from experienced professionals. This exposure has helped me clarify my career aspirations, refine my skills, and develop a clearer understanding of the industry landscape.

How will this experience impact your next semester at Lubin?

My internship at MetLife will likely bring a deeper understanding of practical application to my studies at Lubin. It may provide real-world examples to enhance my coursework, potential networking opportunities with professionals in my field, and help me refine my career goals by gaining experience in finance and accounting.

Do you have any advice for other Lubin students who would like to pursue an internship in the future?

Networking plays a crucial role—it's not just about connecting with established professionals, but also fellow interns who could become a future partner of big companies. Building relationships during internships can open doors to valuable opportunities and provide insight into your chosen field.

Connect with Emiliah:

Contains Video
No
Subscribe to