You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 15:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun May 17, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24560992726,1cyclictest28-21ksoftirqd/212:23:012
2456099270,26cyclictest0-21swapper/209:40:592
2455699260,1cyclictest4530-21ssh10:52:331
2456299250,24cyclictest17953-21diskmemload10:50:013
24560992523,0cyclictest151rcu_preempt11:33:192
2456099250,24cyclictest17953-21diskmemload10:44:042
2456099250,24cyclictest0-21swapper/211:59:432
2455699250,2cyclictest16488-21apt10:35:001
2455699250,24cyclictest6442-21apt-get12:22:221
2455699250,0cyclictest17953-21diskmemload09:30:341
24552992524,1cyclictest25037-21rm12:09:020
24552992523,1cyclictest9047-21ssh12:23:290
2455299250,24cyclictest18119-21ssh09:38:250
2455299250,24cyclictest17953-21diskmemload10:48:380
24562992422,1cyclictest22017-21idleruntime-cro10:40:013
24562992422,1cyclictest0-21swapper/310:27:233
2456299240,23cyclictest17953-21diskmemload12:00:243
2456299240,23cyclictest17953-21diskmemload11:21:113
2456299240,1cyclictest6407-21/usr/sbin/munin12:22:263
2456299240,0cyclictest0-21swapper/310:19:503
24560992423,1cyclictest28-21ksoftirqd/211:27:492
24560992422,1cyclictest0-21swapper/210:25:462
2456099240,24cyclictest0-21swapper/211:20:362
2456099240,24cyclictest0-21swapper/209:50:212
2456099240,23cyclictest17953-21diskmemload12:08:272
2456099240,23cyclictest17953-21diskmemload10:31:252
2456099240,23cyclictest0-21swapper/210:21:322
24556992422,1cyclictest0-21swapper/111:12:101
2455699240,24cyclictest17953-21diskmemload10:38:091
2455699240,23cyclictest30967-21ssh10:18:361
2455699240,23cyclictest0-21swapper/110:57:221
24552992420,3cyclictest0-21swapper/010:42:230
2455299240,23cyclictest26921-21munin-run07:15:000
2455299240,23cyclictest17953-21diskmemload10:30:090
2455299240,0cyclictest6392-21ssh10:54:590
24562992323,0cyclictest0-21swapper/312:30:223
24562992323,0cyclictest0-21swapper/310:33:493
24562992323,0cyclictest0-21swapper/310:14:403
24562992323,0cyclictest0-21swapper/309:34:553
24562992321,1cyclictest0-21swapper/312:37:563
24562992321,1cyclictest0-21swapper/312:05:063
2456299230,2cyclictest27928-21ssh11:42:323
2456299230,2cyclictest2347-21ssh10:51:503
2456299230,23cyclictest0-21swapper/311:49:323
2456299230,23cyclictest0-21swapper/311:16:063
2456299230,23cyclictest0-21swapper/309:56:033
2456299230,23cyclictest0-21swapper/309:42:163
2456299230,23cyclictest0-21swapper/309:28:343
2456299230,22cyclictest479-21sh10:20:543
2456299230,22cyclictest3356-21ssh11:50:183
2456299230,22cyclictest29277-21apt09:20:013
2456299230,0cyclictest17953-21diskmemload12:14:263
2456299230,0cyclictest0-21swapper/312:16:583
2456299230,0cyclictest0-21swapper/309:50:083
2456299230,0cyclictest0-21swapper/309:46:333
24560992323,0cyclictest17953-21diskmemload11:49:242
24560992323,0cyclictest17953-21diskmemload10:13:272
24560992323,0cyclictest0-21swapper/212:39:582
24560992323,0cyclictest0-21swapper/212:17:252
24560992323,0cyclictest0-21swapper/210:56:352
24560992323,0cyclictest0-21swapper/209:49:522
24560992321,1cyclictest0-21swapper/210:50:392
24560992321,1cyclictest0-21swapper/209:58:202
24560992321,1cyclictest0-21swapper/209:32:322
24560992320,2cyclictest30750-21sh10:47:272
2456099231,21cyclictest313-21systemd-journal07:55:002
2456099230,23cyclictest0-21swapper/211:10:192
2456099230,23cyclictest0-21swapper/210:35:282
2456099230,23cyclictest0-21swapper/210:15:122
2456099230,22cyclictest31018-21ssh09:22:022
2456099230,22cyclictest17953-21diskmemload12:30:492
2456099230,22cyclictest10381-21sh12:25:052
2456099230,1cyclictest28-21ksoftirqd/211:09:232
2456099230,0cyclictest0-21swapper/212:02:582
2456099230,0cyclictest0-21swapper/211:50:562
2456099230,0cyclictest0-21swapper/211:16:012
24556992323,0cyclictest0-21swapper/112:16:411
24556992323,0cyclictest0-21swapper/112:11:361
24556992323,0cyclictest0-21swapper/111:56:561
24556992323,0cyclictest0-21swapper/110:42:161
24556992323,0cyclictest0-21swapper/110:00:171
24556992321,1cyclictest0-21swapper/112:35:391
2455699230,2cyclictest25195-21ssh12:09:131
2455699230,23cyclictest17953-21diskmemload11:09:021
2455699230,23cyclictest0-21swapper/111:38:551
2455699230,23cyclictest0-21swapper/110:29:381
2455699230,22cyclictest7968-21ssh11:25:241
2455699230,22cyclictest30489-21ssh11:45:391
2455699230,22cyclictest26141-21ssh09:46:281
2455699230,22cyclictest17953-21diskmemload11:02:251
2455699230,22cyclictest14751-21ssh09:36:081
2455699230,22cyclictest1151-21diskmemload10:21:421
2455699230,22cyclictest0-21swapper/112:30:361
2455699230,0cyclictest17953-21diskmemload11:15:111
2455699230,0cyclictest0-21swapper/109:53:131
24552992323,0cyclictest0-21swapper/012:04:570
24552992323,0cyclictest0-21swapper/010:15:160
24552992323,0cyclictest0-21swapper/010:12:540
24552992323,0cyclictest0-21swapper/009:47:150
24552992323,0cyclictest0-21swapper/009:13:590
24552992322,1cyclictest17953-21diskmemload11:13:280
24552992322,1cyclictest0-21swapper/012:15:430
24552992321,1cyclictest13-21ksoftirqd/011:50:310
24552992321,1cyclictest0-21swapper/010:22:240
2455299230,2cyclictest32745-21ssh11:18:030
2455299230,2cyclictest21649-21sh12:35:180
2455299230,23cyclictest0-21swapper/010:09:230
2455299230,23cyclictest0-21swapper/007:22:220
2455299230,22cyclictest8377-21ssh09:30:070
2455299230,22cyclictest7754-21sh11:25:090
2455299230,22cyclictest6196-21ssh10:25:560
2455299230,22cyclictest0-21swapper/011:45:390
2455299230,22cyclictest0-21swapper/010:35:260
2455299230,0cyclictest687-21smartctl09:52:250
2455299230,0cyclictest17953-21diskmemload10:01:490
2455299230,0cyclictest17953-21diskmemload09:58:260
2455299230,0cyclictest0-21swapper/012:28:040
2455299230,0cyclictest0-21swapper/009:28:120
24562992222,0cyclictest17953-21diskmemload09:35:133
24562992222,0cyclictest0-21swapper/311:25:433
24562992221,1cyclictest17953-21diskmemload11:10:223
24562992220,1cyclictest11642-21ssh10:59:353
24562992220,1cyclictest0-21swapper/309:20:143
24562992219,1cyclictest313-21systemd-journal07:40:013
2456299220,22cyclictest0-21swapper/312:26:443
2456299220,22cyclictest0-21swapper/311:05:463
2456299220,22cyclictest0-21swapper/309:10:063
2456299220,21cyclictest26906-21latency_hist07:15:003
2456299220,21cyclictest1540-21apt-get08:37:203
2456299220,21cyclictest10036-21ssh10:00:463
2456299220,21cyclictest0-21swapper/307:47:213
2456299220,1cyclictest20481-21apt-get11:37:223
2456299220,1cyclictest17953-21diskmemload11:56:443
2456299220,1cyclictest17366-21sh11:04:413
2456299220,1cyclictest0-21swapper/311:31:053
2456299220,1cyclictest0-21swapper/310:41:323
2456299220,1cyclictest0-21swapper/307:20:003
24560992222,0cyclictest0-21swapper/211:41:502
24560992221,1cyclictest28-21ksoftirqd/211:03:142
24560992220,1cyclictest9125-21grep07:45:012
24560992220,1cyclictest0-21swapper/209:10:312
2456099220,21cyclictest21126-21perf08:10:002
2456099220,21cyclictest0-21swapper/211:36:162
2456099220,21cyclictest0-21swapper/209:25:422
2456099220,1cyclictest606-21dbus-daemon08:20:012
2456099220,1cyclictest18738-21munin-run08:05:002
2456099220,1cyclictest14993-21ssh09:36:272
2456099220,1cyclictest0-21swapper/209:15:262
24556992222,0cyclictest0-21swapper/109:20:021
24556992221,1cyclictest2764-21ssh09:25:061
24556992220,1cyclictest15631-21cron12:30:011
24556992219,2cyclictest18471-21ssh09:10:201
2455699220,21cyclictest24979-21ssh11:40:531
2455699220,21cyclictest0-21swapper/109:55:371
2455699220,21cyclictest0-21swapper/109:40:261
2455699220,1cyclictest3803-21apt-get11:22:221
2455699220,1cyclictest17953-21diskmemload10:45:261
2455699220,1cyclictest15212-21sh12:00:321
2455699220,1cyclictest0-21swapper/111:30:141
2455699220,1cyclictest0-21swapper/110:05:381
2455699220,0cyclictest16768-21apt-get08:02:221
2455699220,0cyclictest0-21swapper/110:12:161
24552992222,0cyclictest0-21swapper/011:57:290
24552992222,0cyclictest0-21swapper/011:04:120
24552992222,0cyclictest0-21swapper/009:16:410
24552992221,1cyclictest1241-21ssh09:23:210
24552992220,1cyclictest19732-21sh09:40:250
24552992220,1cyclictest17981-21cat09:10:010
2455299220,22cyclictest0-21swapper/011:07:240
2455299220,21cyclictest3522-21awk08:40:000
2455299220,1cyclictest7464-21ssh10:56:110
2455299220,1cyclictest4882-21ssh11:22:270
2455299220,1cyclictest27349-21apt-get07:17:210
2455299220,1cyclictest26543-21ssh12:10:460
2455299220,0cyclictest17953-21diskmemload12:31:090
2455299220,0cyclictest17953-21diskmemload11:30:270
24562992121,0cyclictest0-21swapper/310:05:433
24562992120,0cyclictest0-21swapper/307:22:223
2456299210,21cyclictest0-21swapper/308:42:223
2456299210,20cyclictest0-21swapper/308:02:223
2456299210,20cyclictest0-21swapper/307:42:223
24560992121,0cyclictest17953-21diskmemload12:10:242
24560992119,1cyclictest0-21swapper/207:35:012
2456099210,20cyclictest0-21swapper/207:30:012
2456099210,20cyclictest0-21swapper/207:12:222
2456099210,0cyclictest0-21swapper/210:00:472
2456099210,0cyclictest0-21swapper/208:32:202
24556992121,0cyclictest0-21swapper/111:50:141
24556992121,0cyclictest0-21swapper/107:37:241
24556992120,1cyclictest26775-21apt-get08:22:221
24556992119,1cyclictest9117-21turbostat07:45:011
24556992119,1cyclictest16334-21awk08:00:001
2455699210,1cyclictest2147-21apt-get07:32:201
2455699210,1cyclictest11207-21apt-get08:57:241
2455699210,0cyclictest1540-21apt-get08:37:201
2455699210,0cyclictest0-21swapper/108:32:201
2455299210,21cyclictest26337-21perf08:20:010
2455299210,21cyclictest0-21swapper/011:40:050
2455299210,21cyclictest0-21swapper/011:36:260
2455299210,21cyclictest0-21swapper/009:02:220
2455299210,0cyclictest3967-21apt-get08:42:220
2455299210,0cyclictest0-21swapper/007:52:210
24562992020,0cyclictest0-21swapper/309:02:213
24562992020,0cyclictest0-21swapper/307:32:203
24562992019,0cyclictest31564-21apt-get08:32:213
2456299200,0cyclictest0-21swapper/307:52:213
24560992020,0cyclictest0-21swapper/209:10:012
24560992020,0cyclictest0-21swapper/208:10:022
24560992020,0cyclictest0-21swapper/207:47:202
24560992020,0cyclictest0-21swapper/207:22:222
24560992017,2cyclictest609-21rs:main2
2456099200,20cyclictest0-21swapper/208:22:222
2456099200,0cyclictest0-21swapper/208:37:202
24556992020,0cyclictest0-21swapper/108:42:221
24556992019,0cyclictest29767-21apt-get07:22:221
24552992020,0cyclictest21952-21apt-get08:12:200
24552992020,0cyclictest14355-21apt-get07:57:240
24552992020,0cyclictest0-21swapper/008:47:210
24552992020,0cyclictest0-21swapper/007:32:200
24562991919,0cyclictest0-21swapper/309:07:213
24562991917,1cyclictest0-21swapper/308:10:003
24560991919,0cyclictest0-21swapper/208:47:212
24560991918,1cyclictest0-21swapper/208:27:212
24556991917,1cyclictest9559-21apt-get07:47:211
2455699190,18cyclictest0-21swapper/108:27:211
2455699190,0cyclictest0-21swapper/107:17:211
24552991919,0cyclictest4594-21apt-get07:37:230
24552991919,0cyclictest0-21swapper/008:57:230
24552991919,0cyclictest0-21swapper/007:42:220
2456099186,11cyclictest28-21ksoftirqd/207:36:552
2456099185,7cyclictest28-21ksoftirqd/208:42:312
24560991818,0cyclictest0-21swapper/208:57:242
24560991818,0cyclictest0-21swapper/208:52:222
24560991816,1cyclictest0-21swapper/209:05:012
24556991818,0cyclictest0-21swapper/108:10:021
24556991818,0cyclictest0-21swapper/107:55:001
24552991818,0cyclictest8811-21apt-get08:52:220
24552991818,0cyclictest29167-21apt-get08:27:210
24552991818,0cyclictest0-21swapper/008:32:200
24552991818,0cyclictest0-21swapper/007:47:200
24552991817,1cyclictest0-21swapper/008:05:000
24562991717,0cyclictest0-21swapper/307:27:203
2456299170,17cyclictest0-21swapper/308:27:203
24556991717,0cyclictest0-21swapper/109:07:201
24556991717,0cyclictest0-21swapper/108:47:211
24562991616,0cyclictest0-21swapper/308:47:213
24552991616,0cyclictest0-21swapper/008:22:220
2456299150,15cyclictest0-21swapper/308:56:313
24560991513,1cyclictest29326-21perf07:20:002
24556991512,2cyclictest26315-21ls08:20:011
24552991515,0cyclictest0-21swapper/008:07:230
24562991414,0cyclictest0-21swapper/308:50:353
24562991414,0cyclictest0-21swapper/308:20:033
2456299140,14cyclictest0-21swapper/308:15:083
2456299140,14cyclictest0-21swapper/308:10:213
2456299140,0cyclictest0-21swapper/307:55:073
2455699140,14cyclictest0-21swapper/109:00:241
2455699140,14cyclictest0-21swapper/107:10:421
2455699140,0cyclictest0-21swapper/108:50:111
2455699140,0cyclictest0-21swapper/107:25:291
2455299140,14cyclictest0-21swapper/007:25:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional