You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-25 - 08:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed Mar 25, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5693992828,0cyclictest0-21swapper/123:24:581
5699992623,2cyclictest9444-21ssh22:55:443
5698992622,4cyclictest31383-21diskmemload22:43:052
5698992621,4cyclictest0-21swapper/222:32:552
569899260,25cyclictest31383-21diskmemload21:46:052
569899260,0cyclictest0-21swapper/200:21:352
5693992624,1cyclictest0-21swapper/122:51:181
569999250,24cyclictest12866-21df_inode00:02:553
5698992520,4cyclictest13880-21apt-get23:00:022
5698992520,4cyclictest0-21swapper/222:46:412
569899250,24cyclictest31383-21diskmemload23:58:312
569899250,24cyclictest0-21swapper/200:07:542
569899250,1cyclictest0-21swapper/223:43:302
5693992523,1cyclictest0-21swapper/100:36:281
569399252,22cyclictest22-21ksoftirqd/122:55:141
569099250,24cyclictest5987-21ssh23:56:020
569999240,24cyclictest0-21swapper/322:05:493
569999240,23cyclictest31383-21diskmemload22:27:473
569999240,23cyclictest0-21swapper/323:55:193
569999240,23cyclictest0-21swapper/321:39:463
569999240,23cyclictest0-21swapper/319:32:523
569999240,0cyclictest0-21swapper/323:30:403
5698992424,0cyclictest0-21swapper/223:20:362
5698992421,3cyclictest0-21swapper/223:31:352
5698992421,3cyclictest0-21swapper/221:26:062
5698992421,2cyclictest31383-21diskmemload00:37:142
5698992420,4cyclictest31383-21diskmemload21:18:302
5698992420,3cyclictest0-21swapper/223:48:232
5698992420,3cyclictest0-21swapper/200:26:302
5698992420,3cyclictest0-21swapper/200:15:302
5698992420,3cyclictest0-21swapper/200:11:142
5698992415,5cyclictest24065-21sh21:34:002
569899240,21cyclictest0-21swapper/222:20:192
569899240,21cyclictest0-21swapper/221:38:472
569899240,20cyclictest0-21swapper/222:51:152
569899240,1cyclictest0-21swapper/223:18:352
569899240,1cyclictest0-21swapper/221:55:382
569899240,0cyclictest0-21swapper/221:24:272
569899240,0cyclictest0-21swapper/219:52:532
5693992424,0cyclictest22-21ksoftirqd/122:15:311
5693992424,0cyclictest0-21swapper/100:20:421
5693992422,2cyclictest22-21ksoftirqd/121:45:011
5693992422,1cyclictest211rcuc/123:26:391
569399240,23cyclictest31383-21diskmemload23:47:241
569399240,23cyclictest25927-21ssh23:12:001
569399240,23cyclictest21301-21ssh23:07:491
569399240,23cyclictest0-21swapper/123:19:131
569399240,23cyclictest0-21swapper/122:33:261
569399240,0cyclictest31383-21diskmemload22:40:391
569399240,0cyclictest31383-21diskmemload21:56:511
5690992423,1cyclictest31383-21diskmemload21:59:500
5690992423,1cyclictest0-21swapper/021:28:310
569099240,24cyclictest31383-21diskmemload23:41:050
569099240,23cyclictest31383-21diskmemload23:03:250
569099240,23cyclictest25186-21rm00:14:370
569099240,23cyclictest0-21swapper/020:53:000
569099240,1cyclictest5525-21ntp_states21:47:580
569099240,1cyclictest32353-21ssh21:11:140
569099240,0cyclictest31383-21diskmemload23:13:450
569099240,0cyclictest2221-21ssh00:23:220
569099240,0cyclictest0-21swapper/023:47:290
569099240,0cyclictest0-21swapper/022:55:140
5699992323,0cyclictest0-21swapper/323:52:053
5699992323,0cyclictest0-21swapper/323:17:013
5699992322,1cyclictest31383-21diskmemload21:41:163
5699992321,1cyclictest20549-21memory22:02:583
5699992320,2cyclictest30433-21ssh00:19:513
569999230,2cyclictest25620-21apt-get22:40:003
569999230,2cyclictest24778-21diskmemload21:35:013
569999230,2cyclictest2197-21ssh21:12:593
569999230,2cyclictest20312-21rm00:09:493
569999230,23cyclictest0-21swapper/322:49:433
569999230,23cyclictest0-21swapper/300:14:573
569999230,22cyclictest8259-21ssh23:26:183
569999230,22cyclictest5688-21ssh22:52:493
569999230,22cyclictest3125-21sh00:24:543
569999230,22cyclictest2685-21ssh21:45:123
569999230,1cyclictest26446-21apt-get23:12:523
569999230,1cyclictest12285-21rm21:54:483
569999230,0cyclictest31383-21diskmemload21:25:143
569999230,0cyclictest0-21swapper/300:31:033
5698992323,0cyclictest0-21swapper/221:42:242
5698992323,0cyclictest0-21swapper/200:02:422
5698992320,3cyclictest0-21swapper/223:35:432
5698992318,4cyclictest0-21swapper/221:51:042
5698992317,2cyclictest0-21swapper/222:15:342
569899230,23cyclictest0-21swapper/223:53:052
569899230,23cyclictest0-21swapper/223:06:322
569899230,23cyclictest0-21swapper/221:14:022
569899230,22cyclictest24195-21perf20:55:002
569899230,22cyclictest0-21swapper/222:00:362
569899230,20cyclictest0-21swapper/200:34:282
569899230,0cyclictest0-21swapper/223:28:302
569899230,0cyclictest0-21swapper/223:11:112
5693992323,0cyclictest31383-21diskmemload22:21:311
5693992323,0cyclictest0-21swapper/123:54:131
5693992323,0cyclictest0-21swapper/123:01:291
5693992323,0cyclictest0-21swapper/121:51:461
5693992323,0cyclictest0-21swapper/121:37:061
5693992321,1cyclictest32652-21ssh22:47:121
5693992321,1cyclictest19596-21apt-get22:02:541
5693992321,1cyclictest10382-21sh21:21:041
5693992320,2cyclictest15421-21ssh21:26:051
569399231,1cyclictest22-21ksoftirqd/100:33:211
569399231,1cyclictest22-21ksoftirqd/100:00:311
569399230,2cyclictest17657-21munin-plugin-st19:35:001
569399230,23cyclictest0-21swapper/121:46:501
569399230,23cyclictest0-21swapper/121:33:411
569399230,22cyclictest13341-21ssh23:31:261
569399230,22cyclictest10955-21rm22:25:271
569399230,22cyclictest0-21swapper/122:06:301
569399230,22cyclictest0-21swapper/100:10:161
569399230,1cyclictest31383-21diskmemload00:26:031
569399230,0cyclictest0-21swapper/123:56:411
569399230,0cyclictest0-21swapper/123:40:361
569399230,0cyclictest0-21swapper/122:12:471
569399230,0cyclictest0-21swapper/121:11:501
5690992323,0cyclictest0-21swapper/022:39:200
5690992323,0cyclictest0-21swapper/021:15:340
5690992323,0cyclictest0-21swapper/000:15:480
5690992322,1cyclictest3580-21kworker/0:122:15:350
5690992322,1cyclictest31383-21diskmemload00:36:010
5690992321,2cyclictest0-21swapper/023:50:560
5690992320,2cyclictest4054-21diskmemload00:26:030
569099230,23cyclictest0-21swapper/023:22:300
569099230,23cyclictest0-21swapper/022:28:360
569099230,23cyclictest0-21swapper/021:32:180
569099230,22cyclictest4363-21ssh22:50:400
569099230,22cyclictest19343-21sh20:45:010
569099230,22cyclictest0-21swapper/021:37:530
569099230,22cyclictest0-21swapper/000:32:380
569099230,0cyclictest31383-21diskmemload22:00:380
569099230,0cyclictest11230-21ssh21:53:020
5699992222,0cyclictest0-21swapper/322:30:393
5699992222,0cyclictest0-21swapper/321:56:173
5699992222,0cyclictest0-21swapper/300:26:463
5699992221,1cyclictest0-21swapper/323:40:513
5699992221,0cyclictest0-21swapper/323:39:013
5699992220,1cyclictest19639-21ssh23:05:123
5699992219,2cyclictest6967-21munin-plugin-st20:20:003
5699992219,2cyclictest28983-21cron21:05:003
569999221,21cyclictest34-21ksoftirqd/323:01:433
569999220,21cyclictest32578-21apt-get20:07:493
569999220,21cyclictest31383-21diskmemload22:10:293
569999220,21cyclictest22926-21apt-get19:47:503
569999220,21cyclictest18116-21apt-get19:37:513
569999220,21cyclictest10805-21sh21:21:423
569999220,21cyclictest0-21swapper/322:20:243
569999220,21cyclictest0-21swapper/321:15:213
569999220,1cyclictest2877-21ssh23:20:463
569999220,1cyclictest26674-21ssh22:40:373
569999220,1cyclictest0-21swapper/322:15:253
569999220,0cyclictest0-21swapper/323:47:083
5698992222,0cyclictest0-21swapper/222:12:152
5698992220,1cyclictest31383-21diskmemload22:25:482
569899220,21cyclictest17437-21apt-get20:42:522
569899220,21cyclictest0-21swapper/222:36:572
569899220,1cyclictest0-21swapper/222:05:192
5693992220,1cyclictest22-21ksoftirqd/120:55:001
569399220,21cyclictest31383-21diskmemload21:15:301
569399220,21cyclictest26152-21ssh00:15:421
569399220,21cyclictest18116-21apt-get19:37:511
569399220,21cyclictest0-21swapper/120:02:551
569399220,1cyclictest0-21swapper/123:37:041
569399220,0cyclictest0-21swapper/120:37:521
5690992222,0cyclictest0-21swapper/023:35:010
5690992222,0cyclictest0-21swapper/022:40:330
5690992220,1cyclictest13046-21rm23:30:520
569099220,2cyclictest30245-21ssh21:40:240
569099220,22cyclictest31383-21diskmemload22:11:130
569099220,22cyclictest0-21swapper/000:05:280
569099220,21cyclictest9926-21ssh21:20:220
569099220,21cyclictest7836-21apt-get20:22:540
569099220,21cyclictest0-21swapper/022:45:130
569099220,21cyclictest0-21swapper/022:20:390
569099220,1cyclictest20311-21ssh23:06:120
569099220,1cyclictest11266-21ssh23:28:240
569099220,1cyclictest0-21swapper/023:15:320
569099220,1cyclictest0-21swapper/022:32:550
569099220,0cyclictest0-21swapper/022:05:010
569099220,0cyclictest0-21swapper/000:04:420
5699992120,0cyclictest0-21swapper/320:12:523
569999210,20cyclictest15019-21apt-get20:37:513
569999210,0cyclictest0-21swapper/320:22:543
569999210,0cyclictest0-21swapper/300:35:473
5698992121,0cyclictest0-21swapper/219:32:512
5698992120,1cyclictest21748-21perf20:50:012
5693992121,0cyclictest0-21swapper/122:35:581
5693992121,0cyclictest0-21swapper/120:47:501
5693992121,0cyclictest0-21swapper/100:05:341
5693992120,1cyclictest10446-21perf19:20:001
5693992119,1cyclictest0-21swapper/120:35:011
569399210,20cyclictest22-21ksoftirqd/120:07:561
569399210,20cyclictest0-21swapper/119:47:511
5699992020,0cyclictest0-21swapper/319:52:523
5699992020,0cyclictest0-21swapper/319:25:003
569999200,0cyclictest0-21swapper/320:57:543
569999200,0cyclictest0-21swapper/319:12:513
5698992019,1cyclictest2614-21apt-get20:12:522
5698992016,4cyclictest0-21swapper/220:27:532
569899200,1cyclictest22595-21latency_hist19:45:012
569899200,1cyclictest22595-21latency_hist19:45:012
569899200,0cyclictest5013-21apt-get20:17:522
5693992020,0cyclictest0-21swapper/120:42:521
5693992020,0cyclictest0-21swapper/119:12:511
569399200,19cyclictest6950-21perf20:20:001
5690992020,0cyclictest0-21swapper/020:17:510
5690992020,0cyclictest0-21swapper/020:07:500
5690992020,0cyclictest0-21swapper/019:37:510
569099200,19cyclictest0-21swapper/019:57:520
569099200,0cyclictest30192-21apt-get20:02:550
569099200,0cyclictest0-21swapper/021:02:520
5698991917,1cyclictest9759-21perf20:25:002
569899190,1cyclictest13237-21expr20:32:542
5693991919,0cyclictest0-21swapper/121:02:521
5693991917,1cyclictest22466-21munin-run19:45:011
5693991917,1cyclictest22466-21munin-run19:45:011
5690991919,0cyclictest0-21swapper/019:42:520
5690991919,0cyclictest0-21swapper/019:42:520
5690991917,1cyclictest21758-21cat20:50:010
569099190,18cyclictest10405-21cron19:20:000
569099190,0cyclictest0-21swapper/019:22:510
5699991818,0cyclictest27773-21apt-get19:57:523
5699991818,0cyclictest10228-21apt-get20:27:533
5699991818,0cyclictest0-21swapper/320:47:503
5699991818,0cyclictest0-21swapper/319:27:533
5699991817,1cyclictest20533-21apt-get19:42:523
5699991817,1cyclictest20533-21apt-get19:42:523
5698991818,0cyclictest0-21swapper/221:02:522
5698991817,0cyclictest24676-21apt-get20:57:542
5698991816,1cyclictest1281-21apt-get20:07:562
5698991816,1cyclictest1-21systemd19:25:002
5698991816,1cyclictest0-21swapper/220:05:002
569899180,0cyclictest0-21swapper/219:47:512
5693991818,0cyclictest27773-21apt-get19:57:511
5690991818,0cyclictest0-21swapper/019:32:520
5690991816,1cyclictest606-21dbus-daemon21:10:000
5699991716,1cyclictest0-21swapper/321:07:513
5699991715,1cyclictest606-21dbus-daemon20:45:013
569899177,9cyclictest0-21swapper/219:37:582
569899170,7cyclictest0-21swapper/219:27:532
5690991717,0cyclictest0-21swapper/020:57:540
5690991717,0cyclictest0-21swapper/020:32:520
5690991716,0cyclictest0-21swapper/019:52:530
5690991713,4cyclictest3580-21kworker/0:119:27:110
5699991614,1cyclictest10459-21awk19:20:003
569899163,8cyclictest16997-21latency_hist20:40:012
569899160,3cyclictest0-21swapper/219:13:012
5693991616,0cyclictest0-21swapper/120:27:531
5693991616,0cyclictest0-21swapper/120:12:521
5693991614,1cyclictest6964-21apt-get20:20:021
5693991614,1cyclictest0-21swapper/120:58:031
5693991614,1cyclictest0-21swapper/119:30:001
5690991614,1cyclictest12154-21perf20:30:010
5698991515,0cyclictest0-21swapper/219:57:522
569899150,15cyclictest0-21swapper/221:05:502
569399150,1cyclictest25619-21cat19:52:531
5690991515,0cyclictest0-21swapper/020:12:520
569099150,4cyclictest0-21swapper/019:47:510
569999140,14cyclictest0-21swapper/320:00:093
569999140,0cyclictest0-21swapper/320:50:133
569999140,0cyclictest0-21swapper/320:30:353
569899140,0cyclictest0-21swapper/219:15:192
569399140,14cyclictest0-21swapper/121:05:111
569399140,14cyclictest0-21swapper/119:20:091
569099140,14cyclictest0-21swapper/020:36:330
569099140,14cyclictest0-21swapper/019:10:530
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional