You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-24 - 11:10
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackeslot2.osadl.org (updated Wed Apr 24, 2024 00:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39441721520,2sleep20-21swapper/220:10:022
46265021140,1sleep20-21swapper/221:05:292
2358491300,2ptp4l0-21swapper/300:05:143
328123992720,5cyclictest0-21swapper/022:25:020
328126992619,6cyclictest682085-21kthreadcore23:25:171
2358491260,2ptp4l0-21swapper/319:25:093
72738422521,2sleep20-21swapper/223:54:202
61667722521,2sleep20-21swapper/222:45:132
328123992520,4cyclictest1511-21rs:main0
2358491250,5ptp4l164790irq/127-enp1s0-TxRx-023:57:533
2358491250,5ptp4l164790irq/127-enp1s0-TxRx-022:33:083
2358491250,5ptp4l164790irq/127-enp1s0-TxRx-022:28:233
2358491250,5ptp4l164790irq/127-enp1s0-TxRx-022:12:293
2358491250,5ptp4l164790irq/127-enp1s0-TxRx-021:51:443
2358491250,5ptp4l164790irq/127-enp1s0-TxRx-000:12:553
2358491250,3ptp4l0-21swapper/319:55:003
2358491250,3ptp4l0-21swapper/319:54:593
77643622420,2sleep20-21swapper/200:25:062
73512122421,2sleep20-21swapper/223:58:242
49411722420,2sleep20-21swapper/221:27:302
3612272240,1chrt361230-21kthreadcore19:40:241
328123992419,4cyclictest623848-21munin-plugin-st22:50:000
328123992418,5cyclictest471416-21apt-key21:15:000
2358491240,5ptp4l164790irq/127-enp1s0-TxRx-023:47:473
2358491240,5ptp4l164790irq/127-enp1s0-TxRx-023:37:323
2358491240,5ptp4l164790irq/127-enp1s0-TxRx-022:40:003
2358491240,5ptp4l164790irq/127-enp1s0-TxRx-021:15:163
2358491240,5ptp4l164790irq/127-enp1s0-TxRx-000:38:293
2358491240,5ptp4l164790irq/127-enp1s0-TxRx-000:18:093
2358491240,4ptp4l164790irq/127-enp1s0-TxRx-023:12:003
2358491240,4ptp4l164790irq/127-enp1s0-TxRx-022:18:053
73645022319,2sleep20-21swapper/200:00:132
71179622319,2sleep20-21swapper/223:44:542
63765122319,2sleep20-21swapper/222:55:312
63193622319,2sleep20-21swapper/222:54:522
61529522319,2sleep20-21swapper/222:43:542
51840822319,2sleep20-21swapper/221:43:132
50919122319,2sleep20-21swapper/221:35:482
328123992318,4cyclictest656035-21systemd23:10:010
2358491230,6ptp4l164790irq/127-enp1s0-TxRx-022:52:403
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-023:41:413
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-023:09:343
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-022:41:213
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-022:01:013
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-021:36:333
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-021:13:353
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-020:00:133
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-019:08:123
2358491230,4ptp4l164790irq/127-enp1s0-TxRx-023:18:133
2358491230,3ptp4l164790irq/127-enp1s0-TxRx-022:07:523
2358491230,10ptp4l164790irq/127-enp1s0-TxRx-023:26:313
79128422218,2sleep20-21swapper/200:33:352
76044422218,2sleep20-21swapper/200:15:082
69492822218,2sleep20-21swapper/223:33:112
68717522218,2sleep20-21swapper/223:28:382
67010422218,2sleep20-21swapper/223:16:312
66391722218,2sleep20-21swapper/223:14:562
58969022218,2sleep20-21swapper/222:25:412
57614622218,2sleep20-21swapper/222:20:132
57515922218,2sleep20-21swapper/222:19:002
55822122218,2sleep20-21swapper/222:08:052
53475422218,2sleep20-21swapper/221:54:042
45700022219,2sleep20-21swapper/221:00:562
328123992217,4cyclictest607888-21ls22:40:000
2577798220,2rtkit-daemon0-21swapper/023:17:460
2358491220,5ptp4l164790irq/127-enp1s0-TxRx-023:50:313
2358491220,5ptp4l164790irq/127-enp1s0-TxRx-023:23:543
2358491220,5ptp4l164790irq/127-enp1s0-TxRx-023:02:143
2358491220,5ptp4l164790irq/127-enp1s0-TxRx-000:31:033
2358491220,4ptp4l164790irq/127-enp1s0-TxRx-022:23:433
2358491220,4ptp4l164790irq/127-enp1s0-TxRx-021:40:133
2358491220,4ptp4l164790irq/127-enp1s0-TxRx-021:02:413
2358491220,3ptp4l164790irq/127-enp1s0-TxRx-000:04:383
2358491220,3ptp4l0-21swapper/322:45:103
2358491220,2ptp4l0-21swapper/320:10:203
44555522118,2sleep20-21swapper/220:51:502
43409922118,2sleep20-21swapper/220:41:442
42844322118,2sleep20-21swapper/220:37:532
41685622118,2sleep20-21swapper/220:26:262
41115022118,2sleep20-21swapper/220:22:012
40500322118,2sleep20-21swapper/220:15:282
38817122118,2sleep20-21swapper/220:02:102
38235222118,2sleep20-21swapper/219:56:292
37089022118,2sleep20-21swapper/219:46:552
36070622118,2sleep20-21swapper/219:40:222
34241022118,2sleep20-21swapper/219:24:382
33350422118,2sleep20-21swapper/219:10:592
328126992117,2cyclictest471800-21munin-run21:15:021
328123992116,4cyclictest680808-21cut23:25:130
328123992116,4cyclictest405802-21systemctl20:20:020
328123992116,4cyclictest0-21swapper/020:30:010
328123992115,4cyclictest1454-1atopacctd22:40:020
2358491210,6ptp4l164790irq/127-enp1s0-TxRx-019:55:133
2358491210,5ptp4l164790irq/127-enp1s0-TxRx-023:30:113
2358491210,5ptp4l164790irq/127-enp1s0-TxRx-020:26:593
2358491210,4ptp4l164790irq/127-enp1s0-TxRx-023:00:013
2358491210,4ptp4l164790irq/127-enp1s0-TxRx-000:25:233
2358491210,4ptp4l164790irq/127-enp1s0-TxRx-000:21:473
2358491210,3ptp4l164790irq/127-enp1s0-TxRx-020:35:133
2358491210,3ptp4l164790irq/127-enp1s0-TxRx-019:30:283
52704222017,1sleep20-21swapper/221:49:352
328126992017,2cyclictest648251-21systemd-run23:05:021
328126992017,2cyclictest623820-21latency_hist22:50:001
328126992017,2cyclictest3103116-1atop20:00:011
32812399203,6cyclictest0-21swapper/021:32:200
32812399202,2cyclictest471rcuog/200:21:050
328123992015,4cyclictest784052-21grep00:30:010
328123992015,4cyclictest463619-21systemd-run21:10:030
328123992015,4cyclictest1511-21rs:main0
328123992014,5cyclictest594934-21kthreadcore22:30:210
2577798200,2rtkit-daemon463077-21diskmemload00:38:340
2577798200,2rtkit-daemon0-21swapper/020:06:140
2577798200,1rtkit-daemon0-21swapper/020:38:460
2577798200,1rtkit-daemon0-21swapper/019:34:190
2577798200,1rtkit-daemon0-21swapper/000:34:210
2358491200,6ptp4l164790irq/127-enp1s0-TxRx-019:35:133
2358491200,5ptp4l164790irq/127-enp1s0-TxRx-020:54:513
2358491200,5ptp4l164790irq/127-enp1s0-TxRx-020:48:153
2358491200,4ptp4l164790irq/127-enp1s0-TxRx-020:30:133
2358491200,4ptp4l164790irq/127-enp1s0-TxRx-019:44:463
2358491200,3ptp4l164790irq/127-enp1s0-TxRx-020:20:133
2358491200,3ptp4l164790irq/127-enp1s0-TxRx-020:15:253
2358491200,2ptp4l164790irq/127-enp1s0-TxRx-021:26:393
2358491200,2ptp4l0-21swapper/321:49:563
2358491200,2ptp4l0-21swapper/321:30:503
2358491200,2ptp4l0-21swapper/320:05:113
2358491200,1ptp4l0-21swapper/321:20:573
2358491200,10ptp4l164790irq/127-enp1s0-TxRx-019:15:263
328126991916,2cyclictest384129-21kthreadcore20:00:161
328126991915,3cyclictest627076-21kthreadcore22:50:201
328126991915,3cyclictest442945-21kthreadcore20:50:211
328126991915,3cyclictest407098-21kernelversion20:20:171
328126991915,3cyclictest356798-21kthreadcore19:35:241
328126991914,3cyclictest0-21swapper/100:10:011
328126991913,5cyclictest610025-21kthreadcore22:40:201
328126991913,5cyclictest529754-21kthreadcore21:50:201
328126991911,7cyclictest0-21swapper/121:45:091
32812399195,2cyclictest0-21swapper/022:56:330
32812399193,6cyclictest0-21swapper/021:51:360
328123991914,4cyclictest724828-21phc2sys23:50:260
328123991914,4cyclictest567788-21apt-key22:15:010
328123991914,4cyclictest559710-21latency_hist22:10:010
328123991914,4cyclictest480181-21cstates21:20:150
328123991914,4cyclictest445820-21idleruntime-cro20:55:010
328123991913,5cyclictest431495-21kthreadcore20:40:210
328123991913,4cyclictest334273-21sh19:15:020
328123991912,6cyclictest253548-21kworker/u8:3+nfsiod22:19:030
32812399191,17cyclictest476824-21ssh21:15:260
2577798190,2rtkit-daemon333900-21munin-node23:39:100
2577798190,2rtkit-daemon1791-21snmpd19:53:360
2577798190,2rtkit-daemon1791-21snmpd19:53:350
2577798190,1rtkit-daemon0-21swapper/019:48:510
2358491190,5ptp4l164790irq/127-enp1s0-TxRx-020:57:443
2358491190,3ptp4l541ktimers/321:56:463
2358491190,3ptp4l164790irq/127-enp1s0-TxRx-019:10:183
328130991814,2cyclictest0-21swapper/221:55:282
328126991815,2cyclictest653578-21unixbench_multi23:05:311
328126991815,2cyclictest353923-21cron19:35:011
328126991815,2cyclictest25693-21cron00:20:001
328126991815,2cyclictest0-21swapper/100:00:011
328126991814,3cyclictest372618-21kthreadcore19:50:211
328126991814,3cyclictest372618-21kthreadcore19:50:211
328126991814,3cyclictest1509-21in:imuxsock21:30:021
328126991812,5cyclictest560352-21fschecks_count22:10:131
328126991812,5cyclictest1451-21avahi-daemon19:15:011
32812399188,5cyclictest491462-21cut21:25:170
32812399186,7cyclictest407336-21tr20:20:180
32812399183,6cyclictest0-21swapper/021:00:340
328123991813,4cyclictest745027-21gltestperf00:05:170
328123991813,4cyclictest705499-21kthreadcore23:40:200
328123991813,4cyclictest0-21swapper/000:15:000
328123991812,5cyclictest736176-21/usr/sbin/munin00:00:010
328123991812,5cyclictest688407-21cpuspeed_turbos23:30:120
328123991812,5cyclictest674974-21kthreadcore23:20:190
328123991812,5cyclictest520569-21irqcore21:45:160
328123991812,5cyclictest336418-21tr19:15:130
328123991812,5cyclictest1464-21dbus-daemon20:00:010
328123991811,5cyclictest385901-21kthreadcore20:00:170
32812399180,2cyclictest471rcuog/222:52:110
2358491180,4ptp4l164790irq/127-enp1s0-TxRx-019:45:483
2358491180,2ptp4l0-21swapper/321:07:343
328130991715,1cyclictest0-21swapper/221:10:112
328130991714,1cyclictest0-21swapper/200:20:212
32812699178,1cyclictest191rcu_preempt23:35:301
328126991714,2cyclictest769524-21cut00:20:191
328126991714,2cyclictest722833-21kthreadcore23:50:231
328126991714,2cyclictest575732-21basename22:20:001
328126991714,2cyclictest556750-21unixbench-2d22:05:271
328126991714,2cyclictest440128-21systemd-run20:50:011
328126991714,2cyclictest1502-21systemd-logind23:50:011
328126991713,3cyclictest661174-21ptp4l23:10:271
328126991713,3cyclictest602946-21kthreadcore22:35:241
328126991711,5cyclictest0-21swapper/121:40:221
328126991710,1cyclictest471rcuog/200:01:171
32812399179,3cyclictest77910-21kworker/u8:2+nfsiod21:55:080
328123991712,4cyclictest664137-21/usr/sbin/munin23:15:020
328123991711,5cyclictest760592-21cpuspeed_turbos00:15:110
328123991711,5cyclictest737517-21irqrtprio00:00:160
328123991711,5cyclictest715169-21kthreadcore23:45:220
328123991711,5cyclictest560634-21kworker/u8:1+nfsiod22:28:120
328123991711,5cyclictest513573-21kthreadcore21:40:220
328123991711,5cyclictest394913-21fschecks_count20:10:130
328123991710,6cyclictest505017-21kernelversion21:35:200
328123991710,3cyclictest345373-21kthreadcore19:25:150
328123991710,3cyclictest253548-21kworker/u8:3+nfsiod22:01:240
328123991710,2cyclictest437056-21kthreadcore20:45:200
2358491170,5ptp4l164790irq/127-enp1s0-TxRx-020:40:263
2358491170,2ptp4l0-21swapper/319:23:113
32813099163,9cyclictest0-21swapper/222:00:102
328130991614,1cyclictest0-21swapper/223:45:222
328130991614,1cyclictest0-21swapper/223:00:222
328130991614,1cyclictest0-21swapper/220:30:162
328130991614,1cyclictest0-21swapper/219:50:192
328130991614,1cyclictest0-21swapper/219:50:192
328130991614,1cyclictest0-21swapper/200:35:222
328130991614,1cyclictest0-21swapper/200:15:012
328130991613,1cyclictest0-21swapper/219:30:142
328130991613,1cyclictest0-21swapper/200:05:182
32812699169,2cyclictest2577798rtkit-daemon23:41:361
328126991613,2cyclictest537548-21kthreadcore21:55:181
328126991613,2cyclictest511406-21munin-plugin-st21:40:011
328126991613,2cyclictest481933-21kthreadcore21:20:221
328126991613,2cyclictest419997-21kthreadcore20:30:161
328126991613,2cyclictest0-21swapper/121:10:021
328126991613,2cyclictest0-21swapper/100:30:011
328126991612,3cyclictest797579-21unixbench-2d00:35:301
328126991612,3cyclictest496981-21irqrtprio21:30:211
328126991612,3cyclictest455226-21kthreadcore21:00:231
328126991612,3cyclictest430874-21kthreadcore20:40:181
328126991612,3cyclictest400364-21cpuspeed_turbos20:15:131
328126991612,3cyclictest340978-21mailstats19:15:241
328126991611,4cyclictest543771-21cpuspeed_turbos22:00:111
32812399169,6cyclictest362541-21kthreadcore19:40:250
32812399169,6cyclictest356798-21kthreadcore19:35:220
32812399169,2cyclictest418616-21kthreadcore20:30:170
32812399166,5cyclictest342065-21sshd19:20:120
328130991513,1cyclictest0-21swapper/223:20:152
328130991513,1cyclictest0-21swapper/223:05:172
328130991513,1cyclictest0-21swapper/220:45:132
328130991513,1cyclictest0-21swapper/219:35:142
328130991513,1cyclictest0-21swapper/219:25:092
328130991512,1cyclictest0-21swapper/221:30:212
328130991511,2cyclictest0-21swapper/222:35:072
32812699156,1cyclictest191rcu_preempt19:26:441
328126991512,2cyclictest689493-21kthreadcore23:30:191
328126991512,2cyclictest668972-21pmu-power23:15:231
328126991512,2cyclictest576102-21/usr/sbin/munin22:20:231
328126991512,2cyclictest1454-1atopacctd00:30:271
328126991512,2cyclictest1451-21avahi-daemon20:30:011
328126991511,3cyclictest476848-21switchtime21:15:281
328126991511,3cyclictest424293-21irqrtprio20:35:181
328126991511,3cyclictest395919-21kthreadcore20:10:181
328126991511,3cyclictest389881-21kthreadcore20:05:181
328126991511,3cyclictest1454-1atopacctd20:55:201
328126991510,1cyclictest2577798rtkit-daemon22:57:321
32813099149,2cyclictest5983302sleep222:32:032
328130991412,1cyclictest0-21swapper/223:35:152
328130991412,1cyclictest0-21swapper/221:20:182
328130991412,1cyclictest0-21swapper/221:15:162
328130991412,1cyclictest0-21swapper/220:10:242
328130991412,1cyclictest0-21swapper/219:15:032
328130991411,1cyclictest0-21swapper/222:10:182
328130991411,1cyclictest0-21swapper/219:10:012
328126991411,2cyclictest672455-21cpuspeed_turbos23:20:121
328126991411,2cyclictest596042-21kthreadcore22:30:211
328126991410,3cyclictest584981-21gltestperf22:25:161
328126991410,3cyclictest368263-21kthreadcore19:45:201
32812399147,2cyclictest1464-21dbus-daemon19:10:010
328130991311,1cyclictest0-21swapper/220:55:122
32812699139,3cyclictest753667-21kthreadcore00:10:161
32812699129,2cyclictest0-21swapper/119:25:011
32812699107,1cyclictest0-21swapper/119:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional