You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 05:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Apr 11, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2314099263,22cyclictest29949-21idleruntime-cro20:30:013
2313299260,25cyclictest22331-21awk20:15:001
2313099260,25cyclictest16794-21cron21:10:000
23140992523,1cyclictest18240-21cron00:20:003
23140992523,1cyclictest16781-21diskmemload22:44:183
2314099250,24cyclictest9240-21ssh00:12:133
2314099250,24cyclictest0-21swapper/323:11:313
2314099250,24cyclictest0-21swapper/321:14:193
2314099250,0cyclictest0-21swapper/323:39:113
23136992523,1cyclictest0-21swapper/221:54:152
2313699250,24cyclictest0-21swapper/222:20:392
2313699250,1cyclictest0-21swapper/221:35:242
23132992523,1cyclictest0-21swapper/121:50:531
23132992522,2cyclictest18517-21sh00:20:021
2313299250,24cyclictest0-21swapper/122:52:181
23130992523,1cyclictest62550irq/127-eno123:14:080
2313099250,24cyclictest16781-21diskmemload00:16:220
23140992423,1cyclictest16781-21diskmemload22:46:513
2314099240,23cyclictest16781-21diskmemload22:02:163
2314099240,23cyclictest16781-21diskmemload00:32:263
2314099240,23cyclictest0-21swapper/322:54:473
23136992423,1cyclictest6994-21ssh21:33:052
23136992420,1cyclictest0-21swapper/221:23:282
2313699240,23cyclictest8891-21turbostat23:40:012
2313699240,23cyclictest16781-21diskmemload22:52:262
2313699240,23cyclictest16781-21diskmemload00:19:342
2313699240,23cyclictest0-21swapper/223:43:122
2313699240,23cyclictest0-21swapper/222:30:552
2313699240,23cyclictest0-21swapper/222:29:572
2313699240,23cyclictest0-21swapper/222:03:062
2313699240,23cyclictest0-21swapper/221:15:052
2313699240,1cyclictest16781-21diskmemload22:46:162
23132992422,1cyclictest5170-21sh00:08:501
23132992422,1cyclictest0-21swapper/121:59:171
23132992421,2cyclictest17800-21sh23:18:001
2313299240,24cyclictest0-21swapper/122:21:251
2313299240,23cyclictest16781-21diskmemload22:37:511
2313299240,23cyclictest16781-21diskmemload00:18:041
2313299240,0cyclictest0-21swapper/123:12:171
23130992424,0cyclictest0-21swapper/023:03:100
23130992423,1cyclictest16781-21diskmemload21:35:060
23130992423,1cyclictest0-21swapper/023:16:340
23130992420,3cyclictest29316-21rm22:57:250
2313099240,24cyclictest0-21swapper/022:34:460
2313099240,23cyclictest4548-21df00:39:190
2313099240,23cyclictest0-21swapper/022:07:200
2313099240,0cyclictest0-21swapper/000:33:200
23140992323,0cyclictest16781-21diskmemload22:21:393
23140992323,0cyclictest0-21swapper/323:50:593
23140992323,0cyclictest0-21swapper/322:55:453
23140992323,0cyclictest0-21swapper/322:05:093
23140992322,1cyclictest16781-21diskmemload23:20:103
23140992321,1cyclictest25807-21ssh00:28:273
23140992321,1cyclictest25807-21ssh00:28:273
23140992320,2cyclictest5748-21ssh23:05:053
2314099230,23cyclictest16781-21diskmemload00:36:163
2314099230,23cyclictest0-21swapper/323:03:133
2314099230,22cyclictest32132-21rm21:24:573
2314099230,22cyclictest31472-21sh23:30:133
2314099230,22cyclictest30982-21ssh21:55:223
2314099230,22cyclictest21688-21fschecks_count00:24:183
2314099230,22cyclictest20817-21ssh22:18:103
2314099230,22cyclictest10710-21sh21:35:403
2314099230,22cyclictest0-21swapper/323:19:033
2314099230,0cyclictest16781-21diskmemload00:07:403
2314099230,0cyclictest0-21swapper/323:46:063
2314099230,0cyclictest0-21swapper/323:40:523
2314099230,0cyclictest0-21swapper/321:54:343
23136992323,0cyclictest0-21swapper/223:13:572
23136992323,0cyclictest0-21swapper/222:37:522
23136992323,0cyclictest0-21swapper/222:11:592
23136992321,1cyclictest0-21swapper/221:46:192
2313699230,23cyclictest0-21swapper/221:13:192
2313699230,22cyclictest4613-21sh00:07:522
2313699230,22cyclictest19149-21apt-get23:50:022
2313699230,22cyclictest0-21swapper/223:56:152
2313699230,1cyclictest0-21swapper/222:06:242
2313699230,0cyclictest16781-21diskmemload22:19:432
2313699230,0cyclictest0-21swapper/223:46:392
2313699230,0cyclictest0-21swapper/223:30:572
2313699230,0cyclictest0-21swapper/200:35:572
23132992323,0cyclictest16781-21diskmemload22:48:561
23132992323,0cyclictest0-21swapper/123:57:531
23132992323,0cyclictest0-21swapper/122:41:081
23132992323,0cyclictest0-21swapper/122:25:021
23132992323,0cyclictest0-21swapper/121:45:031
23132992323,0cyclictest0-21swapper/121:26:171
23132992322,1cyclictest0-21swapper/122:10:371
23132992321,1cyclictest3821-21ssh22:00:241
23132992321,1cyclictest0-21swapper/123:42:351
23132992321,1cyclictest0-21swapper/121:36:051
23132992321,1cyclictest0-21swapper/121:31:151
23132992320,2cyclictest29462-21ssh00:31:101
23132992320,2cyclictest27864-21ssh21:20:551
2313299230,2cyclictest4819-21ssh23:36:261
2313299230,2cyclictest30700-21chrt00:01:031
2313299230,23cyclictest16781-21diskmemload23:29:581
2313299230,22cyclictest18487-21ssh21:12:171
2313299230,22cyclictest16781-21diskmemload23:52:191
2313299230,22cyclictest16781-21diskmemload23:05:501
2313299230,22cyclictest0-21swapper/122:08:381
2313299230,22cyclictest0-21swapper/121:15:021
2313299230,20cyclictest0-21swapper/100:11:151
2313299230,1cyclictest15759-21sh21:40:341
2313299230,1cyclictest0-21swapper/123:32:411
2313299230,0cyclictest0-21swapper/100:35:191
2313299230,0cyclictest0-21swapper/100:28:371
2313299230,0cyclictest0-21swapper/100:28:371
23130992323,0cyclictest16781-21diskmemload00:03:520
23130992323,0cyclictest0-21swapper/023:53:050
23130992323,0cyclictest0-21swapper/022:42:240
23130992323,0cyclictest0-21swapper/022:36:060
23130992323,0cyclictest0-21swapper/022:04:560
23130992323,0cyclictest0-21swapper/021:49:290
23130992323,0cyclictest0-21swapper/021:40:420
23130992323,0cyclictest0-21swapper/021:11:020
23130992323,0cyclictest0-21swapper/020:04:180
23130992323,0cyclictest0-21swapper/000:28:230
23130992323,0cyclictest0-21swapper/000:28:230
23130992321,1cyclictest21220-21rm00:24:160
23130992321,1cyclictest0-21swapper/021:34:190
2313099230,23cyclictest27633-21ssh23:27:210
2313099230,23cyclictest0-21swapper/023:47:190
2313099230,23cyclictest0-21swapper/023:44:310
2313099230,23cyclictest0-21swapper/023:38:550
2313099230,23cyclictest0-21swapper/022:16:280
2313099230,23cyclictest0-21swapper/000:06:380
2313099230,22cyclictest27909-21latency_hist19:20:000
2313099230,22cyclictest22216-21sh21:15:070
2313099230,22cyclictest0-21swapper/022:52:100
2313099230,22cyclictest0-21swapper/021:29:510
2313099230,22cyclictest0-21swapper/021:21:570
2313099230,1cyclictest31339-21ssh21:55:530
2313099230,0cyclictest0-21swapper/023:58:210
23140992222,0cyclictest0-21swapper/323:57:313
23140992222,0cyclictest0-21swapper/323:25:413
23140992222,0cyclictest0-21swapper/321:46:513
23140992222,0cyclictest0-21swapper/321:40:133
23140992222,0cyclictest0-21swapper/300:01:593
23140992221,1cyclictest22175-21ssh21:15:043
23140992220,1cyclictest8777-21ssh22:37:193
23140992220,1cyclictest0-21swapper/322:35:013
2314099220,2cyclictest30342-21ssh22:26:133
2314099220,22cyclictest14226-21ssh22:10:433
2314099220,21cyclictest0-21swapper/319:29:183
2314099220,0cyclictest16781-21diskmemload21:29:163
23136992222,0cyclictest0-21swapper/222:55:062
23136992222,0cyclictest0-21swapper/221:56:042
23136992222,0cyclictest0-21swapper/200:26:392
23136992222,0cyclictest0-21swapper/200:26:392
23136992220,1cyclictest7738-21apt-get20:54:172
23136992220,1cyclictest21498-21ssh23:20:372
2313699220,22cyclictest0-21swapper/221:40:262
2313699220,21cyclictest3091-21perf19:35:012
2313699220,21cyclictest16781-21diskmemload23:00:452
2313699220,21cyclictest0-21swapper/222:42:132
2313699220,21cyclictest0-21swapper/219:14:152
2313699220,21cyclictest0-21swapper/200:00:132
2313699220,1cyclictest7149-21perf20:50:002
2313699220,1cyclictest0-21swapper/221:25:022
2313699220,1cyclictest0-21swapper/200:32:552
2313699220,1cyclictest0-21swapper/200:14:522
2313699220,0cyclictest0-21swapper/223:29:372
2313699220,0cyclictest0-21swapper/223:15:492
23132992220,1cyclictest19139-21diskmemload22:15:251
23132992220,1cyclictest0-21swapper/123:48:361
2313299220,21cyclictest313-21systemd-journal20:40:011
2313299220,21cyclictest27574-21perf20:25:011
2313299220,21cyclictest17566-21munin-run20:05:001
2313299220,21cyclictest16781-21diskmemload22:55:081
2313299220,21cyclictest0-21swapper/120:44:171
2313299220,21cyclictest0-21swapper/120:34:181
2313299220,21cyclictest0-21swapper/120:29:171
2313299220,1cyclictest15156-21awk20:00:001
2313299220,1cyclictest0-21swapper/123:00:371
23130992222,0cyclictest0-21swapper/019:14:160
23130992221,1cyclictest26084-21ssh21:50:390
23130992221,1cyclictest22336-21turbostat.cron20:15:000
2313099220,22cyclictest0-21swapper/023:22:410
2313099220,22cyclictest0-21swapper/022:10:550
2313099220,22cyclictest0-21swapper/000:10:180
2313099220,21cyclictest5875-21rm23:05:170
2313099220,21cyclictest30305-21latency_hist19:25:000
2313099220,21cyclictest25418-21ssh22:22:110
2313099220,21cyclictest22931-21apt-get20:19:170
2313099220,1cyclictest62550irq/127-eno122:45:470
2313099220,1cyclictest31630-21ssh23:30:380
2314099210,21cyclictest15744-21apt-get20:04:183
2314099210,20cyclictest0-21swapper/321:04:173
2314099210,0cyclictest0-21swapper/321:32:253
2313699210,20cyclictest0-21swapper/220:19:172
2313699210,0cyclictest28146-21apt-get20:29:172
23132992119,1cyclictest19955-21wc20:10:001
23132992118,2cyclictest30295-21sh19:25:001
2313299210,21cyclictest0-21swapper/122:32:161
2313299210,1cyclictest12012-21perf21:00:001
2313299210,0cyclictest0-21swapper/119:49:161
23130992121,0cyclictest0-21swapper/020:29:170
23130992119,1cyclictest4734-21idleruntime-cro20:45:010
2313099210,21cyclictest0-21swapper/022:25:420
2313099210,0cyclictest10185-21apt-get20:59:180
23140992020,0cyclictest0-21swapper/320:54:173
23140992020,0cyclictest0-21swapper/320:19:173
23140992020,0cyclictest0-21swapper/320:14:173
23140992020,0cyclictest0-21swapper/319:45:013
23140992018,1cyclictest27871-21cron19:20:003
23140992018,1cyclictest12009-21cut21:00:003
2314099200,20cyclictest0-21swapper/319:39:173
23136992020,0cyclictest0-21swapper/221:04:172
23132992020,0cyclictest0-21swapper/119:39:171
23132992019,0cyclictest0-21swapper/119:19:181
2313299200,20cyclictest0-21swapper/121:09:181
23130992020,0cyclictest18138-21apt-get20:09:170
23130992020,0cyclictest1266-21apt-get19:34:180
23130992020,0cyclictest0-21swapper/020:54:170
23130992020,0cyclictest0-21swapper/020:49:170
23130992020,0cyclictest0-21swapper/020:24:180
2313099201,19cyclictest4065-21kworker/0:021:02:180
2313099200,0cyclictest0-21swapper/019:54:180
23140991919,0cyclictest0-21swapper/319:49:173
23140991919,0cyclictest0-21swapper/319:24:163
23140991918,1cyclictest25752-21apt-get20:24:183
23140991916,2cyclictest25513-21awk19:15:003
23140991916,2cyclictest2356-21lockfile-create20:40:013
23136991919,0cyclictest0-21swapper/219:44:182
23136991918,0cyclictest477-21apt-get20:39:172
23136991918,0cyclictest3674-21apt-get19:39:182
23136991917,1cyclictest606-21dbus-daemon20:25:012
23136991917,1cyclictest16822-21grep21:10:012
23136991917,1cyclictest0-21swapper/219:25:022
23132991919,0cyclictest0-21swapper/121:04:171
23132991919,0cyclictest0-21swapper/120:49:171
23132991919,0cyclictest0-21swapper/120:19:171
23132991918,0cyclictest31292-21apt-get19:29:181
2313099193,16cyclictest4065-21kworker/0:019:41:340
23130991919,0cyclictest0-21swapper/019:25:020
23136991818,0cyclictest0-21swapper/219:54:182
23136991817,0cyclictest0-21swapper/219:49:172
23136991816,1cyclictest10185-21apt-get20:59:182
23130991818,0cyclictest0-21swapper/019:39:180
23130991816,1cyclictest0-21swapper/020:35:010
2313099181,16cyclictest4065-21kworker/0:020:36:220
23140991717,0cyclictest0-21swapper/321:10:003
23140991717,0cyclictest0-21swapper/320:09:173
2314099171,15cyclictest7030-21head20:49:273
23136991716,1cyclictest28501-21apt-get19:24:152
23140991616,0cyclictest0-21swapper/319:34:183
23132991616,0cyclictest0-21swapper/119:34:181
23140991514,1cyclictest30547-21apt-get20:34:183
23140991512,2cyclictest15138-21idleruntime-cro20:00:003
2314099150,14cyclictest10935-21apt-get19:54:173
23136991515,0cyclictest0-21swapper/220:14:182
2313699150,14cyclictest14969-21tail19:59:262
2313699150,14cyclictest0-21swapper/220:04:272
2314099140,0cyclictest0-21swapper/320:40:073
23136991414,0cyclictest0-21swapper/220:40:392
2313699140,14cyclictest0-21swapper/220:30:142
2313699140,14cyclictest0-21swapper/220:05:222
2313699140,14cyclictest0-21swapper/219:15:062
2313299140,14cyclictest0-21swapper/119:54:361
2313299140,14cyclictest0-21swapper/119:40:071
2313299140,0cyclictest0-21swapper/120:51:361
2313099140,14cyclictest0-21swapper/019:55:260
2313099140,0cyclictest0-21swapper/019:45:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional