You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 08:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8.osadl.org (updated Mon Nov 24, 2025 00:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2512518993026,1cyclictest0-21swapper/321:30:013
2512518992219,1cyclictest0-21swapper/323:45:193
2512518992217,2cyclictest2679762-21sleep21:20:003
27499162210,10sleep02749924-21ssh22:00:370
2512518992017,1cyclictest0-21swapper/321:55:253
251251899201,16cyclictest2564783-21turbostat19:54:553
251251899201,14cyclictest2896147-21chrt23:35:003
251251899199,6cyclictest28795262sleep323:23:513
2512518991916,1cyclictest0-21swapper/319:35:003
2512518991916,1cyclictest0-21swapper/300:40:003
2512518991915,1cyclictest0-21swapper/322:55:013
2512518991915,1cyclictest0-21swapper/319:15:003
251251899191,14cyclictest2586040-21taskset20:05:173
251251899191,13cyclictest2872056-21turbostat23:20:003
2512518991815,1cyclictest0-21swapper/321:00:003
2512518991815,1cyclictest0-21swapper/300:05:203
251251899181,13cyclictest2925471-21taskset23:50:233
251251899181,13cyclictest2539749-21turbostat19:30:013
2512518991715,1cyclictest0-21swapper/320:20:193
2512518991714,1cyclictest0-21swapper/323:25:153
2512518991714,1cyclictest0-21swapper/322:45:013
2512518991714,1cyclictest0-21swapper/320:15:143
251251899171,13cyclictest2656871-21chrt21:00:233
251251899171,12cyclictest2942804-21chrt00:02:503
251251899171,12cyclictest2824210-21taskset22:49:553
251251899171,12cyclictest2791985-21taskset22:29:563
251251899171,12cyclictest2776301-21taskset22:20:153
251251899171,12cyclictest2710113-21chrt21:35:453
251251899171,12cyclictest2663890-21sleep21:14:543
251251899171,12cyclictest2589781-21chrt20:10:113
251251899171,12cyclictest2546493-21taskset19:35:123
2512518991710,4cyclictest2583386-21turbostat20:05:013
251251899170,13cyclictest2744078-21chrt22:00:023
251251899168,5cyclictest2532926-21taskset19:20:293
251251899167,6cyclictest29893152sleep300:30:283
251251899167,6cyclictest27661462sleep322:11:133
2512518991613,1cyclictest0-21swapper/323:12:533
2512518991613,1cyclictest0-21swapper/320:54:443
251251899161,12cyclictest2975552-21taskset00:25:013
251251899161,12cyclictest2972905-21taskset00:20:213
251251899161,12cyclictest2965555-21taskset00:15:393
251251899161,12cyclictest2958068-21chrt00:11:523
251251899161,12cyclictest2935644-21taskset23:59:483
251251899161,12cyclictest2910698-21chrt23:42:173
251251899161,12cyclictest2903980-21turbostat23:40:003
251251899161,12cyclictest2848899-21taskset23:05:153
251251899161,12cyclictest2839371-21taskset22:58:013
251251899161,12cyclictest2798333-21taskset22:31:013
251251899161,12cyclictest2773320-21taskset22:15:213
251251899161,12cyclictest2728186-21taskset21:50:093
251251899161,12cyclictest2728186-21taskset21:50:083
251251899161,12cyclictest2686651-21taskset21:22:353
251251899161,12cyclictest2663510-21taskset21:05:293
251251899161,12cyclictest2552650-21taskset19:40:123
251251899161,12cyclictest2526897-21taskset19:18:213
251251899161,11cyclictest2758205-21taskset22:06:203
251251899161,11cyclictest2608381-21taskset20:25:123
251251899160,13cyclictest2620199-21chrt20:31:063
251251899160,13cyclictest2571474-21chrt19:55:153
147598160,7rtkit-daemon2758356-21ssh22:06:380
27238912150,4sleep12725159-21memory21:45:181
2512518991512,1cyclictest0-21swapper/319:45:223
251251899151,12cyclictest2807102-21chrt22:37:173
251251899151,11cyclictest2845492-21taskset23:00:173
251251899151,11cyclictest2720211-21chrt21:45:113
251251899151,11cyclictest2718173-21taskset21:41:033
251251899151,11cyclictest2702198-21taskset21:31:093
251251899151,11cyclictest2638724-21taskset20:45:293
251251899151,11cyclictest2632575-21taskset20:40:513
269681821410,2sleep10-21swapper/121:30:161
251251899140,10cyclictest2620759-21taskset20:35:063
26196072131,9sleep02619665-21ntp_states20:30:190
147598130,9rtkit-daemon2679494-21ssh21:19:290
147598130,3rtkit-daemon2768118-21apt-get22:15:120
28942242120,4sleep02894232-21ssh23:30:580
28653102120,9sleep12866247-21grep23:15:161
26397122120,8sleep12639332-21/usr/sbin/munin20:50:161
26142952121,3sleep12614341-21munin-run20:30:011
147598120,6rtkit-daemon730-21ntpd00:34:552
29832662110,9sleep02983283-21latency_hist00:30:010
29191192111,3sleep22919126-21sh23:48:142
28315922110,8sleep12831600-21ssh22:53:351
27428592111,8sleep22742865-21ssh21:57:552
25725322111,7sleep12573937-21cat19:55:181
25650202110,8sleep02565054-21apt-get19:50:130
25508252110,1sleep10-21swapper/119:35:171
147598110,7rtkit-daemon720-21snmpd20:23:140
147598110,7rtkit-daemon2596677-21cut20:15:180
147598110,6rtkit-daemon476-21wpa_supplicant21:35:122
147598110,2rtkit-daemon214-21plymouthd23:24:002
29648122100,7sleep22964915-21ntpq00:15:232
29450712101,2sleep00-21swapper/000:05:200
28865472104,1sleep0171rcu_preempt23:26:360
27910532100,6sleep12791061-21ssh22:27:421
27193052100,3sleep12719313-21ssh21:43:431
27042012101,4sleep12704238-21apt-get21:35:121
26936902100,7sleep12693698-21ssh21:25:311
25408632102,5sleep12541823-21kthreadcore19:30:181
25273042101,6sleep22527337-21apt-get19:20:122
25127072101,6sleep12512740-21apt-get19:10:141
147598100,6rtkit-daemon2974463-21sh00:22:592
147598100,6rtkit-daemon214-21plymouthd21:11:250
147598100,6rtkit-daemon1540-21sendmail-mta21:21:141
147598100,6rtkit-daemon1540-21sendmail-mta20:56:480
147598100,5rtkit-daemon2563921-21ntp_states19:45:210
147598100,4rtkit-daemon2887506-21ssh23:28:462
147598100,3rtkit-daemon2474811-21munin-node19:07:100
147598100,3rtkit-daemon214-21plymouthd21:43:282
2942353290,6sleep12942359-21ssh00:01:531
2846374290,6sleep22846381-21sh23:00:452
2809409290,6sleep02810291-21kthreadcore22:40:160
2802321290,7sleep00-21swapper/022:35:200
2774913290,6sleep12774917-21ssh22:18:001
2589171291,5sleep22583624-21/usr/sbin/munin20:05:252
14759890,5rtkit-daemon2996701-21/usr/sbin/munin00:35:202
14759890,5rtkit-daemon22110-21packagekitd20:52:530
14759890,4rtkit-daemon2909377-21ntp_states23:40:202
14759890,4rtkit-daemon2545547-21sed19:30:250
14759890,3rtkit-daemon2832747-21df_inode22:55:152
14759890,2rtkit-daemon214-21plymouthd23:59:282
14759890,2rtkit-daemon214-21plymouthd23:04:200
2993014280,5sleep02994907-21kthreadcore00:35:180
2984273280,2sleep00-21swapper/000:30:160
2973035285,1sleep0221rcuc/000:20:220
2911313284,2sleep00-21swapper/023:43:440
2910311283,2sleep12910317-21ssh23:41:211
2856454281,4sleep02856486-21apt-get23:10:130
2821816281,4sleep02821948-21python322:45:210
2767741281,4sleep12767754-21cron22:15:011
2761430283,3sleep00-21swapper/022:10:190
2741954280,6sleep12741962-21ssh21:55:441
2677918280,3sleep2214-21plymouthd21:15:432
2651749281,3sleep12651784-21apt-get21:00:141
2608346281,4sleep22608378-21apt-get20:25:122
2607236281,4sleep12607282-21ps20:20:231
2596338280,5sleep12596859-21irqrtprio20:15:181
2589796281,4sleep22589818-21apt-get20:10:152
2584001281,4sleep12584929-21sed20:05:161
2571209281,3sleep02571242-21apt-get19:55:130
2546822281,4sleep22547623-21cut19:35:152
2532466281,4sleep12532591-21python319:20:241
2512394281,3sleep30-21swapper/319:10:013
2506563281,2sleep12506585-21apt-get19:05:141
14759880,5rtkit-daemon1540-21sendmail-mta21:02:140
14759880,4rtkit-daemon720-21snmpd00:11:501
14759880,4rtkit-daemon313-21systemd-udevd22:23:430
14759880,4rtkit-daemon313-21systemd-udevd20:47:430
14759880,4rtkit-daemon2981848-21ssh00:26:402
14759880,4rtkit-daemon295-21systemd-journal22:36:422
14759880,4rtkit-daemon2902046-21ssh23:35:322
14759880,4rtkit-daemon2848439-21ssh23:05:040
14759880,4rtkit-daemon2790279-21ssh22:25:550
14759880,4rtkit-daemon2733565-21python321:50:232
14759880,4rtkit-daemon2733565-21python321:50:222
14759880,4rtkit-daemon2545964-21apt-get19:35:000
14759880,4rtkit-daemon2518856-21munin-node20:10:560
14759880,4rtkit-daemon214-21plymouthd23:17:160
14759880,4rtkit-daemon214-21plymouthd00:03:570
14759880,4rtkit-daemon1-21systemd19:19:000
14759880,3rtkit-daemon2912198-21/usr/sbin/munin23:45:181
14759880,2rtkit-daemon214-21plymouthd23:50:191
14759880,1rtkit-daemon0-21swapper/223:31:032
14759880,1rtkit-daemon0-21swapper/020:27:250
2983643270,4sleep12983807-21diskstats00:30:131
2957674270,1sleep00-21swapper/000:10:410
2952125271,3sleep22952247-21hddtemp_smartct00:10:192
2913548270,4sleep02914922-21grep23:45:160
2896602270,1sleep00-21swapper/023:35:140
2872460271,3sleep12872591-21cut23:20:161
2824193271,4sleep1214-21plymouthd22:49:551
2807185270,4sleep12775270-21kworker/u16:1+rpciod22:37:281
2782988270,1sleep20-21swapper/222:23:152
2726472270,1sleep00-21swapper/021:46:500
2726304270,4sleep22726310-21ssh21:46:192
2694043270,5sleep02663874-21diskmemload21:26:180
2687286270,1sleep00-21swapper/021:24:060
2673444274,1sleep10-21swapper/121:15:171
2620768271,3sleep12620800-21apt-get20:35:141
2533800271,3sleep22533833-21apt-get19:25:142
2521024270,3sleep12521057-21apt-get19:15:121
14759870,4rtkit-daemon1540-21sendmail-mta20:06:090
14759870,4rtkit-daemon1-21systemd00:09:311
14759870,4rtkit-daemon0-21swapper/222:30:382
14759870,3rtkit-daemon729-21lldpd20:01:030
14759870,3rtkit-daemon2825315-21sed22:50:162
14759870,3rtkit-daemon2811741-21cut22:40:172
14759870,3rtkit-daemon2620962-21diskstats20:35:150
14759870,3rtkit-daemon214-21plymouthd19:20:170
14759870,3rtkit-daemon214-21plymouthd19:11:300
14759870,3rtkit-daemon1540-21sendmail-mta00:15:031
14759870,3rtkit-daemon1-21systemd23:55:251
14759870,3rtkit-daemon1-21systemd23:20:110
14759870,3rtkit-daemon1-21systemd22:47:062
14759870,3rtkit-daemon1-21systemd21:07:340
14759870,3rtkit-daemon1-21systemd19:43:410
14759870,2rtkit-daemon214-21plymouthd21:59:160
14759870,2rtkit-daemon214-21plymouthd19:26:130
2997275261,2sleep10-21swapper/100:35:271
2973847260,4sleep12847343-21kworker/u16:1+rpciod00:21:331
2950619260,1sleep20-21swapper/200:07:422
2930028260,3sleep02931998-21kthreadcore23:55:180
2927281260,1sleep00-21swapper/023:53:500
2902459260,1sleep10-21swapper/123:36:311
2890010260,3sleep12891957-21cut23:30:181
2870494260,4sleep22870502-21ssh23:16:252
2857808260,3sleep12859490-21grep23:10:181
2851033261,2sleep10-21swapper/123:05:181
2838747260,1sleep00-21swapper/022:56:360
2836944260,3sleep12837593-21wc22:55:201
2815323260,1sleep10-21swapper/122:42:531
2799517260,1sleep10-21swapper/122:33:481
2791273260,1sleep20-21swapper/222:28:112
2776114261,2sleep10-21swapper/122:20:141
2758578260,1sleep10-21swapper/122:07:101
2758386261,3sleep22758394-21ssh22:06:432
2750776260,1sleep20-21swapper/222:02:342
2735337260,1sleep00-21swapper/021:53:520
2735337260,1sleep00-21swapper/021:53:510
2728545261,2sleep10-21swapper/121:50:161
2728545261,2sleep10-21swapper/121:50:151
2711257260,1sleep00-21swapper/021:38:320
2694049260,1sleep20-21swapper/221:26:202
2669354261,3sleep20-21swapper/221:10:212
2663721260,1sleep20-21swapper/221:08:132
2658123260,3sleep12658235-21fschecks_time21:05:151
2657502260,1sleep20-21swapper/221:02:472
2651296260,1sleep20-21swapper/220:57:472
2638956260,1sleep20-21swapper/220:48:302
2632803260,1sleep10-21swapper/120:43:471
2631142260,4sleep22631877-21ls20:40:192
2622123261,2sleep20-21swapper/220:35:192
2620334260,1sleep20-21swapper/220:32:502
2595467260,1sleep10-21swapper/120:11:531
2581053260,3sleep12582330-21memory20:00:201
2570888260,1sleep20-21swapper/219:54:162
2541827261,3sleep22543845-21tr19:30:182
2508151261,2sleep20-21swapper/219:05:192
14759860,3rtkit-daemon730-21ntpd20:40:350
14759860,3rtkit-daemon720-21snmpd21:32:002
14759860,3rtkit-daemon1-21systemd00:00:082
2927109250,3sleep20-21swapper/223:53:282
2880402250,3sleep12880496-21expr23:25:141
2853053250,2sleep20-21swapper/223:05:192
2846305250,2sleep10-21swapper/123:00:361
2824617250,3sleep02824775-21df_inode22:50:130
2798382250,3sleep00-21swapper/022:31:140
2768055251,2sleep22663874-21diskmemload22:15:092
2744097250,3sleep10-21swapper/122:00:061
2718210250,3sleep00-21swapper/021:41:090
2702920251,2sleep00-21swapper/021:32:480
2671154250,3sleep10-21swapper/121:13:281
2615956250,3sleep12617926-21cut20:30:161
2602340251,2sleep22602409-21cut20:20:162
2583271250,3sleep20-21swapper/220:03:492
2570847250,1sleep10-21swapper/119:53:431
2558330250,3sleep10-21swapper/119:41:521
2556537250,3sleep20-21swapper/219:40:212
2522269250,2sleep20-21swapper/219:15:172
2982999240,3sleep10-21swapper/100:29:251
2962308240,2sleep02964325-21grep00:15:210
2863011240,2sleep22863019-21ssh23:12:282
2766812240,2sleep22663874-21diskmemload22:12:532
2685863240,2sleep22663513-21kworker/u16:3+rpciod21:20:472
2650703240,2sleep12650799-21switchtime20:55:221
2640653240,2sleep22641897-21kthreadcore20:50:172
2638757240,3sleep10-21swapper/120:45:561
2597246240,2sleep22598314-21kthreadcore20:15:192
2571394240,2sleep22571472-21fschecks_time19:55:152
2564759240,2sleep2214-21plymouthd19:49:582
2564447240,2sleep10-21swapper/119:45:561
2539576240,3sleep10-21swapper/119:28:051
2513636240,2sleep20-21swapper/219:10:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional