Path: ...!news.mixmin.net!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: marc nicole <mk1853387@gmail.com>
Newsgroups: comp.lang.python
Subject: How to check whether lip movement is significant using face landmarks
 in dlib?
Date: Sat, 5 Oct 2024 19:55:36 +0200
Lines: 36
Message-ID: <mailman.36.1728150919.3018.python-list@python.org>
References: <CAGJtH9Sz8MsB=O1Woc_PiKdaX+6A6G3-avDDXs=BeFb3eagf4w@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-Trace: news.uni-berlin.de /JhnFjFMlsOXU7ALUysS5gqiQ30d0llqI0IuZkyMTmkw==
Cancel-Lock: sha1:mAxhQv7ZHPLyXBEH7L1EBTa4irY= sha256:mj+75Q+qfTqKsPJpNglkOdUoCZnTuI3pA0RqC8tMCWY=
Return-Path: <mk1853387@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
 reason="2048-bit key; unprotected key"
 header.d=gmail.com header.i=@gmail.com header.b=cCWRUF+o;
 dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.115
X-Spam-Level: *
X-Spam-Evidence: '*H*': 0.80; '*S*': 0.03; 'def': 0.04; '(to': 0.07;
 'mar': 0.07; 'telling': 0.07; 'aspect': 0.09; 'compute': 0.09;
 'mouth': 0.09; 'url:master': 0.09; 'url:github': 0.14; '64,':
 0.16; 'inspired': 0.16; 'ratio': 0.16; 'to:addr:python-list':
 0.20; 'closed': 0.22; 'to:no real name:2**1': 0.22; 'subject:How':
 0.23; 'subject:using': 0.23; '<pre': 0.26; 'function': 0.27;
 'sfxlen:2': 0.31; '(as': 0.32; 'message-id:@mail.gmail.com': 0.32;
 'able': 0.34; 'received:google.com': 0.34; 'trying': 0.35;
 'following': 0.35; 'from:addr:gmail.com': 0.35; 'using': 0.37;
 'two': 0.39; 'mentioned': 0.39; 'skip:u 20': 0.39; 'wrote': 0.39;
 'define': 0.40; 'try': 0.40; 'tell': 0.60; 'skip:i 20': 0.62;
 'between': 0.63; 'primary': 0.67; 'movement': 0.69;
 'subject:check': 0.69; 'reach': 0.69; '<span': 0.76; 'skip:y 10':
 0.76; 'href=""': 0.84; 'conclude': 0.84; 'distances': 0.84;
 'indexes': 0.84; 'landmark': 0.84; 'lips': 0.84; 'skip:s 280':
 0.84; 'subject: \n ': 0.84; 'noreferrer"': 0.91; 'assess': 0.93
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1728150917; x=1728755717; darn=python.org;
 h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
 :date:message-id:reply-to;
 bh=EHGcq0HW+8Iqg9BPA+hBm30iUzSAWkF1qrv1pcgieFE=;
 b=cCWRUF+ock3DdBDwlYZQlaCmNhqIcXEn8++9CY7Ol9/M14NcmPgjQm3DgWp757BXOZ
 XlwXR77paZgmeJiv5NyIwJ+Z83Ex9Hk53XaLYhOo3etj0SkyYjIE3iH8NB1RgpZ+mWE6
 UV5PCPS+8Vg9KuvxAL4awLAB68jMWG2K0AuErQ7jvyuZCdem1gFs2IWbFSHXcA6x8NVr
 Ih19AP/GTJhvgMwVlXx+5Z5uIp2IJMOZmJc/BL2qPBht1wkd6qsQ+iwtTgKobsyN7mFN
 mBYRuJ8CTzFfXMjPCY4x/0uA/d3STsBnFj7xSG076UpduB7IKA9xwTOKgExJ+EvCZ4n8
 vpzg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1728150917; x=1728755717;
 h=to:subject:message-id:date:from:mime-version:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=EHGcq0HW+8Iqg9BPA+hBm30iUzSAWkF1qrv1pcgieFE=;
 b=Myjgaxfui8r6cANyM2ywr6u23FfKzFGZ9E8Cm37DGkT5msgwyFj4MYJcqT+o8SyG/g
 bAo/yemHYJSXU1Ajmd3vjtMGM40O1almrCJBoIBABlAljdHHJvRGJDcLDPJe+NX6aRI8
 diXKzySqYg9+Hd5yHgIQEn+Ll2K8IW3KCR1Nk9w9DJQ0UkRSJ+TZlm2HYDThUxVNH50F
 IdS8gTfeCNXtCG4RMpninfp/0aFuuQHxDv1vieNUzpuSkwlCB1oclV8d2ZF07soDdswl
 MmTzjrcgGf3NK6VFPIGwSD3MXHV97hPngzPYJMtXabf03qxCvsPq2bAajg0D6nYqVekn
 9AcA==
X-Forwarded-Encrypted: i=1;
 AJvYcCUMRRimsl4oH5S8RQAdB4wGvvLo7ejrHaWMrzDsXMN0G+jgZGx/UR9Qp91aiO4wcG0Bc+EVEA==@python.org
X-Gm-Message-State: AOJu0YxF1tqyxMCwsjtcQp3uQAjwNmCZIR5vPlJdDnW+7PI8h5dM+wu/
 6dANymZ7EYjn1BuBxlykLAJW7gqvqGEkOqRFAzi9p0uGm1MUfubOIEbDMpO8RBGa3R+m33tEFnz
 Z0Vb2PL1mN5dUJVrIglOkmtG3+hm+sdYu
X-Google-Smtp-Source: AGHT+IFxtOB7AfittaWM/6Vosr7hvLzwOTGAiDwU2C34+WBXntRZuFKz/Q9+RjnP06nWEnkqT+CC3YYMxu+f3p9UxAw=
X-Received: by 2002:a05:690c:3487:b0:6db:da26:e8c6 with SMTP id
 00721157ae682-6e2c72b656bmr50716077b3.44.1728150916735; Sat, 05 Oct 2024
 10:55:16 -0700 (PDT)
X-Content-Filtered-By: Mailman/MimeDel 2.1.39
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
 <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
 <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
 <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <CAGJtH9Sz8MsB=O1Woc_PiKdaX+6A6G3-avDDXs=BeFb3eagf4w@mail.gmail.com>
Bytes: 6175

I am trying to assess whether the lips of a person are moving too much
while the mouth is closed (to conclude they are chewing).

I try to assess the lip movement through landmarks (dlib) :

Inspired by the mouth example (
https://github.com/mauckc/mouth-open/blob/master/detect_open_mouth.py#L17),
and using it before the following function (as a primary condition for
telling the person is chewing), I wrote the following function:

def lips_aspect_ratio(shape):
    # grab the indexes of the facial landmarks for the lip
    (mStart, mEnd) = (61, 68)
    lip = shape[mStart:mEnd]
    print(len(lip))
    # compute the euclidean distances between the two sets of
    # vertical lip landmarks (x, y)-coordinates
    # to reach landmark 68 I need to get lib[7] not lip[6] (while I
get lip[7] I get IndexOutOfBoundError)
    A = dist.euclidean(lip[1], lip[6])  # 62, 68
    B = dist.euclidean(lip[3], lip[5])  # 64, 66

    # compute the euclidean distance between the horizontal
    # lip landmark (x, y)-coordinates
    C = dist.euclidean(lip[0], lip[4])  # 61, 65

    # compute the lip aspect ratio
    mar = (A + B) / (2.0 * C)

    # return the lip aspect ratio
    return mar


How to define an aspect ratio for the lips to conclude they are moving
significantly? Is the mentioned function able to tell whether the lips
are significantly moving while the mouth is closed?