You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 18:30
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot8.osadl.org (updated Fri Nov 21, 2025 12:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6039302330,1sleep30-21swapper/310:50:163
288100992824,2cyclictest579133-21turbostat10:34:593
288100992617,6cyclictest684197-21sleep11:40:003
288100992219,1cyclictest0-21swapper/309:00:003
288100992118,1cyclictest0-21swapper/312:00:003
288100992118,1cyclictest0-21swapper/309:50:223
28810099211,15cyclictest441896-21taskset09:10:003
288100992017,1cyclictest0-21swapper/309:30:143
288100992016,1cyclictest0-21swapper/311:05:003
288100992016,1cyclictest0-21swapper/310:40:003
288100991916,1cyclictest0-21swapper/311:23:083
288100991916,1cyclictest0-21swapper/307:35:003
28810099191,14cyclictest571027-21chrt10:29:563
28810099191,13cyclictest334442-21turbostat07:45:013
28810099191,13cyclictest302952-21taskset07:20:013
28810099187,7cyclictest5614592sleep310:21:193
28810099182,12cyclictest765808-21taskset12:30:143
28810099182,12cyclictest365919-21turbostat08:09:593
288100991815,1cyclictest0-21swapper/312:25:153
288100991815,1cyclictest0-21swapper/309:35:173
288100991815,1cyclictest0-21swapper/308:15:143
288100991815,1cyclictest0-21swapper/308:00:013
28810099181,13cyclictest649375-21taskset11:15:203
28810099181,13cyclictest649375-21taskset11:15:193
28810099181,13cyclictest397373-21turbostat08:35:003
28810099181,13cyclictest347017-21turbostat07:55:013
28810099177,7cyclictest441419-21turbostat09:10:003
288100991714,1cyclictest0-21swapper/309:55:113
28810099171,13cyclictest780590-21taskset12:37:403
28810099171,13cyclictest641980-21chrt11:10:513
28810099171,12cyclictest465765-21taskset09:24:563
28810099171,12cyclictest328089-21taskset07:39:573
28810099170,13cyclictest706268-21taskset11:50:233
28810099170,13cyclictest690773-21taskset11:41:293
3409892162,4sleep1341022-21apt-get07:50:141
28810099166,6cyclictest4559362sleep309:15:513
288100991614,1cyclictest0-21swapper/310:15:173
288100991614,1cyclictest0-21swapper/310:00:173
288100991613,1cyclictest0-21swapper/312:20:173
288100991613,1cyclictest0-21swapper/312:00:233
288100991613,1cyclictest0-21swapper/308:35:183
28810099161,12cyclictest746795-21taskset12:15:233
28810099161,12cyclictest739113-21taskset12:10:473
28810099161,12cyclictest732153-21taskset12:08:333
28810099161,12cyclictest675991-21taskset11:34:483
28810099161,12cyclictest490287-21chrt09:40:033
28810099161,12cyclictest471989-21taskset09:25:323
28810099161,12cyclictest410193-21taskset08:45:043
28810099161,12cyclictest409718-21taskset08:42:083
28810099161,12cyclictest384487-21taskset08:21:363
28810099161,12cyclictest314553-21taskset07:25:183
28810099161,11cyclictest659964-21sleep11:29:553
28810099161,11cyclictest593792-21taskset10:41:213
28810099161,11cyclictest498462-21chrt09:45:113
5380312150,8sleep0538039-21ssh10:08:500
4162072151,4sleep11-21systemd08:50:001
28810099151,12cyclictest538169-21taskset10:09:083
28810099151,11cyclictest698509-21taskset11:45:423
28810099151,11cyclictest633772-21chrt11:05:393
28810099151,11cyclictest617562-21taskset10:55:333
28810099151,11cyclictest596062-21taskset10:45:153
28810099151,11cyclictest434214-21chrt09:00:183
28810099151,11cyclictest359236-21taskset08:00:283
28810099151,11cyclictest308977-21taskset07:23:293
28810099151,10cyclictest416561-21taskset08:50:133
28810099150,12cyclictest544666-21chrt10:10:243
147598150,2rtkit-daemon0-21swapper/211:21:202
28810099141,11cyclictest371868-21taskset08:10:543
28810099141,11cyclictest293993-21taskset07:10:353
28810099141,10cyclictest334674-21taskset07:45:063
28810099140,11cyclictest385008-21taskset08:25:083
147598130,9rtkit-daemon493174-21grep09:40:192
147598130,8rtkit-daemon553708-21ssh10:17:050
7520232123,6sleep1754040-21cut12:20:181
7420792121,8sleep0743016-21cut12:15:170
7251962121,8sleep1725263-21ssh12:05:171
5717982121,8sleep00-21swapper/010:30:180
4669472120,9sleep2467823-21cut09:25:162
147598120,8rtkit-daemon779817-21ssh12:35:540
147598120,8rtkit-daemon720-21snmpd08:55:060
7815762111,6sleep2781650-21unin-run12:40:002
5705552110,8sleep1720-21snmpd10:28:511
5636672111,7sleep2563842-21interrupts10:25:152
4745732113,6sleep00-21swapper/009:30:150
4557912110,8sleep0455799-21ssh09:15:310
3372402110,1sleep10-21swapper/107:45:151
2816942111,5sleep1281727-21apt-get07:05:131
147598110,7rtkit-daemon668527-21fschecks_count11:30:132
6910482100,6sleep2691056-21ssh11:42:052
5796672100,6sleep0579875-21hddtemp_smartct10:35:150
5794452100,2sleep10-21swapper/110:35:121
5228992100,7sleep1522698-21/usr/sbin/munin10:00:131
3538642100,7sleep2354022-21/usr/sbin/munin08:00:162
2945902100,8sleep0295945-21grep07:15:020
147598100,6rtkit-daemon587895-21fw_packets10:40:170
147598100,6rtkit-daemon529748-21ssh10:03:200
147598100,6rtkit-daemon1540-21sendmail-mta08:10:570
147598100,6rtkit-daemon1540-21sendmail-mta07:09:372
147598100,5rtkit-daemon733130-21apt-get12:10:132
147598100,3rtkit-daemon720-21snmpd09:05:240
147598100,3rtkit-daemon545409-21ssh10:12:130
147598100,1rtkit-daemon0-21swapper/007:45:080
755570290,7sleep2755578-21ssh12:20:442
706110290,7sleep2706258-21timerwakeupswit11:50:232
628757290,2sleep10-21swapper/111:05:161
611475290,6sleep2611661-21latency_hist10:55:012
580434291,5sleep20-21swapper/210:35:172
548613290,6sleep2549887-21kthreadcore10:15:172
537987290,1sleep20-21swapper/210:08:442
328376291,4sleep1328409-21apt-get07:40:161
327804290,6sleep1327853-21file07:38:271
315408292,4sleep0214-21plymouthd07:30:010
286167291,5sleep0286691-21grep07:05:200
14759890,6rtkit-daemon720-21snmpd07:31:180
14759890,6rtkit-daemon684101-21ssh11:39:531
14759890,5rtkit-daemon519687-21memory09:55:182
14759890,5rtkit-daemon476-21wpa_supplicant08:36:000
14759890,5rtkit-daemon294500-21munin-node08:06:440
14759890,4rtkit-daemon612008-21df_inode10:55:130
14759890,4rtkit-daemon555731-21hddtemp_smartct10:20:150
14759890,4rtkit-daemon447027-21memory09:10:170
14759890,1rtkit-daemon0-21swapper/011:07:050
14759890,1rtkit-daemon0-21swapper/008:01:580
14759890,1rtkit-daemon0-21swapper/007:35:470
734427280,6sleep1735536-21kthreadcore12:10:181
725035281,4sleep0725193-21df_inode12:05:160
723496281,4sleep2723504-21ssh12:02:212
715153280,1sleep00-21swapper/011:56:500
702950280,5sleep1704956-21tr11:50:181
684705281,4sleep0684854-21grep11:40:150
662970284,2sleep20-21swapper/211:25:182
645504281,5sleep2646651-21cut11:15:162
645504281,5sleep2646651-21cut11:15:152
643739281,4sleep2643753-21cron11:15:002
595255281,4sleep1595263-21ssh10:44:541
562290281,4sleep1562300-21sshd10:23:211
520223280,2sleep0520388-21vmstat09:55:230
506513281,3sleep1506552-21apt-get09:50:111
473262281,4sleep0473266-21ssh09:28:330
450174281,4sleep2450380-21hddtemp_smartct09:15:142
449505280,1sleep10-21swapper/109:14:371
417436280,6sleep2418655-21cat08:50:162
416463281,4sleep1416496-21apt-get08:50:111
384743280,1sleep20-21swapper/208:24:532
378736281,3sleep0378769-21apt-get08:20:140
303067281,3sleep0303099-21apt-get07:20:110
287964280,2sleep30-21swapper/307:09:563
14759880,5rtkit-daemon1540-21sendmail-mta07:56:410
14759880,4rtkit-daemon636756-21sed11:10:140
14759880,4rtkit-daemon625009-21cut11:00:220
14759880,4rtkit-daemon481254-21ssh09:33:252
14759880,4rtkit-daemon294500-21munin-node08:44:490
14759880,4rtkit-daemon214-21plymouthd09:22:502
14759880,4rtkit-daemon1-21systemd09:45:092
14759880,4rtkit-daemon1-21systemd09:20:291
14759880,3rtkit-daemon765801-21apt-get12:30:140
14759880,3rtkit-daemon230105-21kworker/2:1-events09:37:202
14759880,3rtkit-daemon214-21plymouthd12:20:360
14759880,3rtkit-daemon214-21plymouthd07:40:200
14759880,1rtkit-daemon0-21swapper/209:54:052
14759880,1rtkit-daemon0-21swapper/008:54:140
14759880,1rtkit-daemon0-21swapper/008:45:420
762824271,3sleep0762907-21ntp_states12:25:190
748355270,1sleep2724081-21kworker/u16:5+xprtiod12:18:322
733554272,3sleep0734425-21irqrtprio12:10:160
714956270,4sleep1714964-21ssh11:56:191
700353271,3sleep0700402-21munin-run11:50:010
676417270,3sleep2676456-21apt-get11:35:122
666060270,1sleep00-21swapper/011:25:270
658078270,5sleep1658083-21sh11:20:401
654045271,4sleep0656010-21grep11:20:190
624606270,3sleep2625003-21sed11:00:222
619468270,4sleep1619547-21latency_hist10:59:591
530754271,2sleep10-21swapper/110:05:141
482222270,4sleep1214-21plymouthd09:35:121
471651271,3sleep1471865-21ssh09:25:221
430739271,3sleep0432698-21grep09:00:170
321365271,1sleep1171rcu_preempt07:31:061
14759870,4rtkit-daemon515-21gmain11:49:002
14759870,4rtkit-daemon515-21gmain07:10:082
14759870,4rtkit-daemon1-21systemd08:33:430
14759870,4rtkit-daemon1-21systemd08:25:590
14759870,3rtkit-daemon730-21ntpd10:46:270
14759870,3rtkit-daemon730-21ntpd10:26:230
14759870,3rtkit-daemon715724-21ssh11:58:082
14759870,3rtkit-daemon526736-21kthreadcore10:00:172
14759870,3rtkit-daemon294500-21munin-node07:51:060
14759870,3rtkit-daemon214-21plymouthd12:04:140
14759870,3rtkit-daemon214-21plymouthd11:45:151
14759870,3rtkit-daemon1-21systemd10:31:322
14759870,3rtkit-daemon1-21systemd09:19:521
14759870,3rtkit-daemon0-21swapper/008:16:420
14759870,2rtkit-daemon214-21plymouthd11:34:211
14759870,2rtkit-daemon214-21plymouthd10:50:040
780322260,1sleep10-21swapper/112:37:071
766089260,1sleep10-21swapper/112:30:161
765364260,1sleep10-21swapper/112:29:531
764099260,4sleep2764107-21ssh12:26:582
732333261,2sleep20-21swapper/212:08:582
723801263,1sleep10-21swapper/112:03:001
707071260,1sleep00-21swapper/011:51:490
683754260,1sleep00-21swapper/011:39:020
650229260,1sleep00-21swapper/011:16:230
650229260,1sleep00-21swapper/011:16:230
644009260,1sleep10-21swapper/111:15:031
644009260,1sleep10-21swapper/111:15:031
635910260,1sleep10-21swapper/111:10:061
622542260,3sleep1624603-21kthreadcore11:00:211
618360260,3sleep2618368-21ssh10:57:242
601678260,1sleep20-21swapper/210:46:072
594991261,3sleep2594995-21ssh10:44:112
571320260,1sleep10-21swapper/110:30:031
545886260,1sleep20-21swapper/210:13:172
545206260,1sleep10-21swapper/110:11:421
515558260,3sleep1516599-21kthreadcore09:55:161
512906260,1sleep00-21swapper/009:51:470
503719261,3sleep0503852-21python309:45:220
497445260,3sleep0497453-21ssh09:43:250
488096260,1sleep00-21swapper/009:35:360
474732260,3sleep1475669-21ps09:30:161
441398260,3sleep2441894-21ssh09:10:012
440482260,3sleep2440559-21ntp_states09:05:242
434837260,1sleep10-21swapper/109:01:301
422755261,2sleep10-21swapper/108:55:131
403566260,1sleep20-21swapper/208:39:072
397189260,1sleep20-21swapper/208:32:582
385042260,3sleep2385093-21grep08:25:132
378281260,1sleep20-21swapper/208:17:472
372906260,3sleep1373846-21irqstats08:15:181
365850260,1sleep20-21swapper/208:09:212
353264260,1sleep10-21swapper/107:59:461
346935260,1sleep20-21swapper/207:54:152
340427260,1sleep20-21swapper/207:46:062
334200260,1sleep20-21swapper/207:42:172
327783260,1sleep20-21swapper/207:38:122
294057260,1sleep00-21swapper/007:11:250
293442260,3sleep1293500-21ntp_states07:10:211
765768250,2sleep20-21swapper/212:30:132
675412250,3sleep00-21swapper/011:33:300
629639251,2sleep20-21swapper/211:05:172
610319250,3sleep1610325-21ssh10:52:191
602017250,3sleep10-21swapper/110:46:501
552689250,3sleep1552949-21unixbench_singl10:15:241
504908250,2sleep172-21kcompactd009:47:031
428640250,3sleep20-21swapper/208:58:152
405319250,2sleep10-21swapper/108:40:181
403914250,3sleep20-21swapper/208:40:122
380137250,3sleep1381775-21ps08:20:191
371491250,3sleep1371647-21timerwakeupswit08:10:211
366551251,2sleep20-21swapper/208:10:142
359269250,1sleep10-21swapper/108:00:551
347252250,2sleep20-21swapper/207:55:102
321145250,2sleep20-21swapper/207:30:232
315245250,3sleep10-21swapper/107:27:571
309353250,2sleep20-21swapper/207:25:102
304498250,2sleep10-21swapper/107:20:161
302600250,3sleep10-21swapper/107:17:361
302494250,3sleep20-21swapper/207:16:162
14759850,2rtkit-daemon0-21swapper/111:40:151
743017240,2sleep1744974-21kthreadcore12:15:181
659933240,2sleep10-21swapper/111:25:011
556853240,2sleep20-21swapper/210:20:172
490364240,2sleep10-21swapper/109:40:141
463880240,2sleep00-21swapper/009:20:290
441060240,1sleep10-21swapper/109:05:391
399042240,2sleep10-21swapper/108:35:181
391720240,2sleep10-21swapper/108:30:171
308959240,2sleep20-21swapper/207:23:172
429047230,1sleep20-21swapper/209:00:022
410192230,1sleep20-21swapper/208:45:042
385001230,1sleep10-21swapper/108:25:041
359837230,1sleep10-21swapper/108:05:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional