You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 13:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Aug 11, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2982599270,26cyclictest10019-21ssh00:19:270
2983699250,2cyclictest313-21systemd-journal21:30:003
2983699250,24cyclictest29974-21ssh21:15:283
2983699250,1cyclictest0-21swapper/323:33:273
2983299250,24cyclictest28586-21ssh23:37:392
29830992524,1cyclictest22-21ksoftirqd/123:09:301
29830992523,1cyclictest0-21swapper/123:10:561
2983099251,0cyclictest151rcu_preempt21:23:451
2983099250,24cyclictest23462-21diskmemload00:04:231
2983099250,1cyclictest23462-21diskmemload00:24:391
29825992524,1cyclictest0-21swapper/000:04:300
2982599250,24cyclictest13948-21ssh00:22:000
2982599250,24cyclictest0-21swapper/021:10:320
2982599250,1cyclictest0-21swapper/023:14:190
29836992422,1cyclictest3935-21ssh23:15:213
2983699240,23cyclictest24723-21ssh00:03:123
2983699240,23cyclictest23462-21diskmemload22:45:213
2983699240,23cyclictest23462-21diskmemload00:22:403
2983699240,23cyclictest0-21swapper/321:10:023
2983699240,1cyclictest28218-21ssh23:37:123
2983699240,1cyclictest23462-21diskmemload23:29:473
29832992422,1cyclictest23462-21diskmemload22:44:462
29832992419,3cyclictest30217-21sh21:15:462
2983299241,22cyclictest0-21swapper/221:29:242
2983299240,23cyclictest23462-21diskmemload22:26:562
2983299240,23cyclictest23462-21diskmemload22:15:102
2983299240,23cyclictest1727-21ssh23:42:332
2983299240,23cyclictest0-21swapper/223:04:292
2983299240,23cyclictest0-21swapper/222:30:582
2983299240,1cyclictest23462-21diskmemload00:35:332
2983299240,1cyclictest13576-21ssh23:53:252
2983299240,1cyclictest0-21swapper/222:21:422
29830992423,1cyclictest0-21swapper/122:34:531
29830992422,1cyclictest221ksoftirqd/121:48:291
29830992422,1cyclictest14720-21turbostat23:25:001
2983099240,23cyclictest28897-21ssh00:06:151
2983099240,0cyclictest0-21swapper/121:51:551
29825992424,0cyclictest0-21swapper/022:43:230
29825992423,1cyclictest23462-21diskmemload23:39:000
29825992423,1cyclictest0-21swapper/023:32:010
29825992422,1cyclictest10102-21ssh21:54:420
29825992422,1cyclictest0-21swapper/023:46:560
29825992422,1cyclictest0-21swapper/023:09:310
2982599240,23cyclictest23462-21diskmemload23:59:420
2982599240,23cyclictest21241-21ssh23:03:160
2982599240,1cyclictest17539-21kworker/u8:122:37:320
2982599240,0cyclictest0-21swapper/022:47:510
29836992323,0cyclictest0-21swapper/323:50:083
29836992323,0cyclictest0-21swapper/321:46:213
29836992323,0cyclictest0-21swapper/300:06:323
29836992321,1cyclictest30851-21ssh21:44:383
29836992321,1cyclictest0-21swapper/322:40:343
29836992321,1cyclictest0-21swapper/322:20:163
29836992321,1cyclictest0-21swapper/321:39:433
2983699230,23cyclictest0-21swapper/322:55:363
2983699230,23cyclictest0-21swapper/300:19:223
2983699230,22cyclictest783-21snmpd23:47:403
2983699230,22cyclictest32411-21sh23:12:403
2983699230,22cyclictest31579-21awk19:15:013
2983699230,22cyclictest3017-21ssh00:12:293
2983699230,22cyclictest23627-21ssh22:07:273
2983699230,22cyclictest23462-21diskmemload22:11:503
2983699230,22cyclictest17455-21ssh22:01:383
2983699230,22cyclictest14909-21ssh21:30:363
2983699230,22cyclictest14909-21ssh21:30:363
2983699230,22cyclictest0-21swapper/323:24:493
2983699230,22cyclictest0-21swapper/321:24:353
2983699230,1cyclictest31189-21ssh00:38:053
2983699230,0cyclictest23462-21diskmemload22:25:473
2983699230,0cyclictest0-21swapper/322:32:133
2983699230,0cyclictest0-21swapper/300:31:173
29832992323,0cyclictest0-21swapper/223:17:352
29832992323,0cyclictest0-21swapper/222:03:132
29832992323,0cyclictest0-21swapper/200:16:292
29832992321,1cyclictest29858-21apt-get23:10:022
29832992321,1cyclictest0-21swapper/222:37:012
29832992321,1cyclictest0-21swapper/221:31:522
29832992321,1cyclictest0-21swapper/221:31:522
2983299230,2cyclictest3667-21ssh00:13:242
2983299230,2cyclictest23567-21ssh23:33:302
2983299230,23cyclictest23462-21diskmemload22:58:562
2983299230,23cyclictest0-21swapper/221:51:422
2983299230,23cyclictest0-21swapper/200:24:152
2983299230,22cyclictest5691-21ssh23:45:192
2983299230,22cyclictest30575-21ssh00:08:392
2983299230,22cyclictest29016-21ssh23:09:332
2983299230,22cyclictest10730-21rm21:55:102
2983299230,22cyclictest0-21swapper/222:46:482
2983299230,1cyclictest0-21swapper/221:40:232
2983299230,0cyclictest0-21swapper/200:31:362
2983299230,0cyclictest0-21swapper/200:27:222
2983299230,0cyclictest0-21swapper/200:01:012
29830992323,0cyclictest23462-21diskmemload00:35:481
29830992323,0cyclictest0-21swapper/123:45:191
29830992323,0cyclictest0-21swapper/123:36:471
29830992323,0cyclictest0-21swapper/121:55:551
29830992323,0cyclictest0-21swapper/121:10:091
29830992323,0cyclictest0-21swapper/100:29:591
29830992321,1cyclictest28015-21ssh21:43:081
29830992321,1cyclictest0-21swapper/123:15:311
29830992321,1cyclictest0-21swapper/120:15:011
29830992321,1cyclictest0-21swapper/100:31:311
29830992320,2cyclictest8113-21ssh22:50:311
29830992320,2cyclictest18644-21rm23:57:451
2983099230,23cyclictest0-21swapper/123:52:561
2983099230,23cyclictest0-21swapper/122:40:581
2983099230,23cyclictest0-21swapper/122:05:151
2983099230,23cyclictest0-21swapper/100:11:401
2983099230,22cyclictest7173-21ssh00:15:271
2983099230,22cyclictest20820-21ls23:30:001
2983099230,22cyclictest17443-21ssh22:01:371
2983099230,22cyclictest14702-21rm22:57:031
2983099230,20cyclictest0-21swapper/119:24:261
2983099230,1cyclictest22051-21apt-get23:04:251
2983099230,1cyclictest211rcuc/123:42:111
2983099230,0cyclictest0-21swapper/123:30:371
2983099230,0cyclictest0-21swapper/122:46:331
2983099230,0cyclictest0-21swapper/122:22:121
2983099230,0cyclictest0-21swapper/122:19:041
2983099230,0cyclictest0-21swapper/121:25:291
29825992323,0cyclictest23462-21diskmemload21:22:230
29825992323,0cyclictest0-21swapper/023:16:050
29825992323,0cyclictest0-21swapper/022:55:480
29825992323,0cyclictest0-21swapper/022:50:060
29825992323,0cyclictest0-21swapper/022:20:110
29825992323,0cyclictest0-21swapper/022:11:000
29825992321,1cyclictest28546-21sh00:05:490
29825992321,1cyclictest23462-21diskmemload00:12:050
29825992321,1cyclictest21939-21ssh22:05:020
29825992320,2cyclictest16158-21sh21:32:200
29825992320,2cyclictest16158-21sh21:32:200
2982599230,23cyclictest0-21swapper/023:28:580
2982599230,23cyclictest0-21swapper/023:20:320
2982599230,23cyclictest0-21swapper/022:00:160
2982599230,23cyclictest0-21swapper/021:59:380
2982599230,22cyclictest6671-21awk20:35:000
2982599230,22cyclictest27709-21ssh21:42:460
2982599230,1cyclictest30561-21ssh21:16:240
2982599230,1cyclictest0-21swapper/023:54:330
2982599230,1cyclictest0-21swapper/022:17:460
2982599230,0cyclictest0-21swapper/023:43:190
2982599230,0cyclictest0-21swapper/000:36:120
29836992222,0cyclictest23462-21diskmemload22:35:113
29836992222,0cyclictest23462-21diskmemload21:53:273
29836992222,0cyclictest0-21swapper/323:02:103
29836992222,0cyclictest0-21swapper/321:56:183
29836992220,1cyclictest317-21apt-get19:19:273
2983699220,21cyclictest27228-21apt-get20:14:253
2983699220,21cyclictest26369-21ssh23:07:403
2983699220,21cyclictest18231-21diskmemload00:25:103
2983699220,21cyclictest0-21swapper/322:15:073
2983699220,21cyclictest0-21swapper/320:44:263
2983699220,1cyclictest32468-21ssh23:40:173
2983699220,1cyclictest2029-21apt-get20:29:233
2983699220,1cyclictest1424-21turbostat20:25:003
2983699220,1cyclictest10453-21sh22:53:563
2983699220,1cyclictest0-21swapper/323:55:403
29832992222,0cyclictest0-21swapper/222:10:362
29832992222,0cyclictest0-21swapper/221:46:212
29832992222,0cyclictest0-21swapper/221:35:422
29832992220,1cyclictest21074-21apt21:05:002
29832992220,1cyclictest0-21swapper/221:11:072
29832992220,1cyclictest0-21swapper/219:30:002
29832992220,0cyclictest0-21swapper/222:07:562
2983299220,2cyclictest17178-21ssh23:55:352
2983299220,21cyclictest8689-21ssh22:51:232
2983299220,21cyclictest0-21swapper/221:20:122
2983299220,1cyclictest10487-21sh23:21:322
2983299220,0cyclictest23462-21diskmemload23:25:022
29830992222,0cyclictest22-21ksoftirqd/121:17:061
29830992222,0cyclictest0-21swapper/121:37:471
29830992222,0cyclictest0-21swapper/121:31:391
29830992222,0cyclictest0-21swapper/121:31:391
2983099220,21cyclictest0-21swapper/120:19:271
2983099220,21cyclictest0-21swapper/119:54:251
2983099220,1cyclictest0-21swapper/122:26:041
2983099220,0cyclictest0-21swapper/122:37:301
29825992222,0cyclictest23462-21diskmemload21:25:430
29825992222,0cyclictest0-21swapper/021:37:340
29825992221,1cyclictest23462-21diskmemload22:26:170
29825992220,1cyclictest21650irq/140-nvme0q119:45:000
2982599220,2cyclictest19851-21ssh00:28:080
2982599220,21cyclictest8032-21apt-get19:34:260
2982599220,1cyclictest1757-21sh21:48:240
2982599220,1cyclictest0-21swapper/000:31:260
29832992121,0cyclictest0-21swapper/220:29:232
2983299210,1cyclictest18702-21sendmail-msp21:00:012
29830992121,0cyclictest0-21swapper/122:10:161
2983099210,21cyclictest0-21swapper/120:49:261
2983099210,21cyclictest0-21swapper/120:09:261
2983099210,20cyclictest0-21swapper/120:34:241
2983099210,20cyclictest0-21swapper/120:04:251
29825992120,0cyclictest0-21swapper/020:24:250
29825992120,0cyclictest0-21swapper/019:59:250
29825992119,1cyclictest17021-21turbostat.cron19:50:000
29825992119,1cyclictest0-21swapper/020:20:000
2982599210,21cyclictest0-21swapper/020:54:250
29836992018,1cyclictest10421-21apt-get19:39:233
29836992018,1cyclictest0-21swapper/319:55:003
2983699200,20cyclictest0-21swapper/320:59:243
2983699200,0cyclictest0-21swapper/319:34:263
29832992019,0cyclictest15232-21apt-get19:49:242
29832992018,1cyclictest606-21dbus-daemon19:55:002
2983299200,0cyclictest0-21swapper/220:39:252
29830992020,0cyclictest20021-21apt-get19:59:251
29830992020,0cyclictest0-21swapper/120:54:251
29830992020,0cyclictest0-21swapper/120:44:261
29830992020,0cyclictest0-21swapper/120:39:251
29830992020,0cyclictest0-21swapper/120:29:231
29830992018,1cyclictest12230-21grep19:40:001
29825992020,0cyclictest0-21swapper/020:49:260
29825992020,0cyclictest0-21swapper/020:14:250
29825992020,0cyclictest0-21swapper/019:19:270
29836991919,0cyclictest7276-21apt-get20:39:253
29836991919,0cyclictest0-21swapper/320:09:263
29832991919,0cyclictest11498-21perf20:45:012
29832991918,1cyclictest12088-21apt-get20:49:262
29830991919,0cyclictest0-21swapper/120:24:251
29830991916,3cyclictest19292-21apt-get21:04:251
29825991919,0cyclictest0-21swapper/021:09:260
29825991918,1cyclictest16875-21apt-get20:59:240
29836991818,0cyclictest0-21swapper/321:04:243
29836991818,0cyclictest0-21swapper/319:44:263
29832991818,0cyclictest0-21swapper/220:04:252
29832991816,1cyclictest12841-21apt-get19:44:252
29832991815,2cyclictest29626-21apt-get20:19:272
29830991818,0cyclictest0-21swapper/120:59:241
29825991818,0cyclictest22437-21apt-get20:04:250
2983699170,14cyclictest0-21swapper/319:23:523
2983299170,16cyclictest4995-21munin-run19:25:012
29830991717,0cyclictest0-21swapper/119:14:251
29830991715,1cyclictest0-21swapper/121:10:001
29825991714,2cyclictest21098-21idleruntime-cro21:05:000
29832991615,1cyclictest32048-21apt-get20:24:252
29832991615,1cyclictest30381-21apt-get19:14:242
29830991616,0cyclictest0-21swapper/119:34:261
29825991616,0cyclictest0-21swapper/019:29:240
2982599160,1cyclictest4991-21perf19:25:010
2982599160,1cyclictest3922-21sed20:30:000
29836991515,0cyclictest0-21swapper/321:09:263
29836991515,0cyclictest0-21swapper/320:04:253
29836991513,1cyclictest21334-21find19:59:323
29832991515,0cyclictest0-21swapper/219:39:232
29830991515,0cyclictest0-21swapper/119:49:241
29825991515,0cyclictest0-21swapper/019:39:230
29836991414,0cyclictest0-21swapper/320:52:283
29836991414,0cyclictest0-21swapper/319:46:373
2983699140,14cyclictest0-21swapper/320:31:373
2983699140,0cyclictest0-21swapper/320:45:443
2983699140,0cyclictest0-21swapper/320:15:113
29832991414,0cyclictest0-21swapper/219:33:192
2983299140,14cyclictest15981-21sed20:54:342
2983299140,14cyclictest0-21swapper/221:05:352
2983299140,14cyclictest0-21swapper/220:10:492
2983299140,14cyclictest0-21swapper/219:15:402
2983299140,0cyclictest3831-21apt-get20:30:012
29830991414,0cyclictest0-21swapper/119:41:241
2983099140,14cyclictest0-21swapper/119:15:151
2983099140,0cyclictest0-21swapper/119:25:111
29825991414,0cyclictest0-21swapper/020:35:020
29825991414,0cyclictest0-21swapper/019:10:140
2982599140,14cyclictest0-21swapper/020:07:540
2982599140,0cyclictest0-21swapper/020:40:120
2982599140,0cyclictest0-21swapper/019:52:510
29836991313,0cyclictest0-21swapper/319:27:323
29832991313,0cyclictest0-21swapper/219:55:302
29832991212,0cyclictest0-21swapper/220:09:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional