You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-18 - 10:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri Sep 18, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3018429369,8sleep20-21swapper/219:07:242
3023429167,8sleep00-21swapper/019:08:020
3038328471,9sleep30-21swapper/319:09:583
3030228463,6sleep10-21swapper/119:08:541
151082550,0sleep20-21swapper/222:21:252
30476992617,6cyclictest32-21ksoftirqd/221:19:572
3047999243,2cyclictest0-21swapper/300:25:143
3047999240,23cyclictest0-21swapper/300:14:393
30479992321,1cyclictest1142-21runrttasks21:28:303
3047999230,1cyclictest0-21swapper/321:38:093
30476992318,4cyclictest32-21ksoftirqd/222:39:592
3047699231,3cyclictest0-21swapper/223:30:572
3047699230,3cyclictest0-21swapper/222:45:272
3047699230,22cyclictest0-21swapper/221:41:022
3047699230,21cyclictest0-21swapper/222:55:142
30479992219,2cyclictest1142-21runrttasks21:46:273
3047499220,0cyclictest0-21swapper/123:25:141
30479992120,1cyclictest0-21swapper/322:41:373
3047699210,0cyclictest0-21swapper/222:32:242
30469992119,1cyclictest1142-21runrttasks00:36:360
30469992118,2cyclictest1142-21runrttasks21:42:580
3046999210,4cyclictest0-21swapper/022:17:260
3047999200,19cyclictest0-21swapper/322:39:363
3047699201,1cyclictest23262-21diskmemload22:16:052
3047499207,2cyclictest225-21systemd-journal20:59:591
30479991916,2cyclictest1142-21runrttasks00:23:173
3047999190,18cyclictest0-21swapper/300:01:513
3047699194,9cyclictest0-21swapper/220:10:132
30476991919,0cyclictest0-21swapper/223:39:412
30476991915,3cyclictest32-21ksoftirqd/219:10:162
30476991914,4cyclictest32-21ksoftirqd/221:00:122
3047699191,1cyclictest141rcu_preempt00:36:412
3047699190,1cyclictest0-21swapper/223:10:012
3047699190,1cyclictest0-21swapper/223:10:002
3047699190,19cyclictest0-21swapper/223:07:572
3047699190,18cyclictest0-21swapper/223:24:542
30474991916,1cyclictest0-21swapper/122:45:141
3047499191,1cyclictest9191-21irqstats21:20:181
30479991816,1cyclictest10528-21aten2_r7power_c00:15:143
3047999181,1cyclictest0-21swapper/323:33:133
3047999180,1cyclictest0-21swapper/323:12:353
3047999180,1cyclictest0-21swapper/323:12:343
3047699184,7cyclictest0-21swapper/222:11:102
30476991814,4cyclictest32-21ksoftirqd/200:10:182
3047699180,18cyclictest0-21swapper/221:26:362
3047699180,16cyclictest0-21swapper/200:33:482
3047499184,2cyclictest141rcu_preempt00:09:181
30474991818,0cyclictest0-21swapper/123:50:331
3047499180,2cyclictest0-21swapper/122:59:291
3047499180,18cyclictest16153-21kworker/1:000:03:231
3047699177,9cyclictest32-21ksoftirqd/223:55:132
3047699171,5cyclictest32-21ksoftirqd/200:01:092
30476991715,1cyclictest1142-21runrttasks23:00:582
30476991712,5cyclictest32-21ksoftirqd/221:53:542
3047699171,1cyclictest141rcu_preempt21:22:582
3047699170,14cyclictest0-21swapper/200:22:092
3046999176,6cyclictest26831-21apt-get22:10:010
3046999174,10cyclictest1037-21gdbus00:07:380
30469991714,2cyclictest7578-21dump-pmu-power23:39:560
99882160,1sleep30-21swapper/322:57:103
30479991615,1cyclictest0-21swapper/323:41:453
30479991615,1cyclictest0-21swapper/322:05:363
30479991614,2cyclictest39-21ksoftirqd/323:57:463
30479991614,1cyclictest29860-21ssh22:50:143
3047999160,2cyclictest0-21swapper/323:08:573
3047699166,2cyclictest141rcu_preempt23:15:212
30476991615,1cyclictest0-21swapper/222:53:152
30476991614,2cyclictest32-21ksoftirqd/200:28:072
3047699161,3cyclictest141rcu_preempt22:25:582
3047699160,1cyclictest0-21swapper/220:03:552
3047699160,16cyclictest0-21swapper/222:41:592
30474991616,0cyclictest0-21swapper/100:30:231
30474991614,1cyclictest0-21swapper/123:20:321
30469991612,3cyclictest23681-21ls23:05:190
3046999160,2cyclictest0-21swapper/021:25:450
3047999158,6cyclictest1037-21gdbus00:07:393
30479991515,0cyclictest0-21swapper/322:24:053
30479991514,1cyclictest0-21swapper/321:52:393
3047999150,14cyclictest0-21swapper/320:55:133
3047699157,3cyclictest32-21ksoftirqd/219:29:062
3047699157,1cyclictest141rcu_preempt21:05:152
30476991514,1cyclictest0-21swapper/220:49:322
30476991510,1cyclictest0-21swapper/200:09:132
30474991514,1cyclictest7257-21kworker/1:123:57:031
30474991511,3cyclictest18655-21wc22:05:011
3047499150,12cyclictest0-21swapper/122:10:131
30469991513,1cyclictest1142-21runrttasks22:35:180
3046999150,13cyclictest18038-21ssh21:45:130
3047999149,3cyclictest22095-21wc23:05:003
30479991410,3cyclictest4229-21cut22:15:173
30479991410,3cyclictest31570-21apache223:21:403
3047699146,6cyclictest32-21ksoftirqd/223:43:282
3047699142,4cyclictest32-21ksoftirqd/219:20:522
3047699142,3cyclictest32-21ksoftirqd/221:55:282
3047699141,2cyclictest26229-21strings21:30:182
3047699140,1cyclictest0-21swapper/200:15:562
3047699140,10cyclictest0-21swapper/220:09:492
3047499148,5cyclictest1318-21sendmail_mailtr21:15:231
30474991412,1cyclictest6857-21ssh23:15:131
30474991412,1cyclictest1142-21runrttasks21:44:381
30474991411,2cyclictest26813-21cron22:10:011
3046999146,7cyclictest1746-21ssh21:55:010
3046999142,1cyclictest141rcu_preempt22:44:290
30469991410,3cyclictest22848-21grep21:05:270
30469991410,2cyclictest8346-21wc20:35:260
3047999139,2cyclictest9793-21grep23:35:253
3047999138,3cyclictest20877-21idleruntime-cro21:05:003
30479991312,1cyclictest0-21swapper/319:25:143
30479991311,1cyclictest549-21latency_hist21:34:593
30479991311,1cyclictest1142-21runrttasks23:25:483
30479991310,2cyclictest5230-21head20:30:163
30479991310,2cyclictest23951-21aten2_r7power_v21:10:153
30479991310,2cyclictest14272-21grep19:40:213
30479991310,2cyclictest12364-21head20:45:173
3047699135,2cyclictest141rcu_preempt20:35:452
3047699134,7cyclictest32-21ksoftirqd/221:10:222
3047699133,7cyclictest10550irq/122-ahci[0019:34:592
30476991311,2cyclictest32-21ksoftirqd/222:00:312
30476991310,2cyclictest3764-21tail20:25:262
30476991310,2cyclictest15703-21head20:50:262
30476991310,2cyclictest1330-21ssh21:35:142
3047699130,1cyclictest141rcu_preempt21:47:522
3047499139,3cyclictest17458-21head21:25:161
3047499139,3cyclictest11723-21grep00:15:211
3047499132,1cyclictest23262-21diskmemload21:14:461
3047499131,7cyclictest225-21systemd-journal21:45:491
30474991312,1cyclictest4570-21kworker/1:022:34:131
30474991310,2cyclictest2518-21cron00:30:001
30474991310,2cyclictest23366-21cat21:09:591
3046999136,6cyclictest31582-21apache221:34:140
30469991310,2cyclictest9515-21tr19:30:210
30469991310,2cyclictest3580-21ls00:10:240
30469991310,2cyclictest29152-21tr22:30:180
30469991310,2cyclictest22417-21wc20:00:010
30469991310,2cyclictest14659-21cron23:20:000
119932130,2sleep30-21swapper/322:00:243
3047999129,2cyclictest22370-21grep22:45:173
3047999129,2cyclictest21411-21grep21:05:153
3047999129,2cyclictest21243-21sed22:25:173
3047999128,3cyclictest3762-21sort00:30:163
3047999128,3cyclictest3039-21cron19:19:593
3047999128,3cyclictest18832-21wc19:50:193
3047999128,3cyclictest18832-21wc19:50:193
3047999128,2cyclictest8971-21sh20:40:003
3047999128,2cyclictest8317-21cut21:20:143
3047999127,5cyclictest23262-21diskmemload21:56:353
3047999123,4cyclictest225-21systemd-journal21:20:003
3047699129,2cyclictest9511-21tr20:40:142
3047699129,2cyclictest32-21ksoftirqd/221:00:002
3047699129,2cyclictest30484-21sed20:15:162
3047699129,2cyclictest25424-21chrt22:08:552
3047699129,2cyclictest2067-21cut23:50:212
30476991210,1cyclictest23262-21diskmemload23:25:362
3047499129,2cyclictest824-21cat23:30:201
3047499129,2cyclictest4284-21sort22:15:171
3047499129,2cyclictest3976-21chrt21:36:121
3047499129,2cyclictest3290-21cat19:20:001
3047499129,2cyclictest20886-21sh21:05:001
3047499129,2cyclictest18570-21grep19:50:161
3047499129,2cyclictest18570-21grep19:50:151
3047499129,2cyclictest177942sleep123:40:231
3047499129,2cyclictest16124-21cut19:45:171
3047499129,2cyclictest12717-21ntpq00:35:211
3047499129,2cyclictest12677-21head22:20:171
3047499128,3cyclictest2432-21hostname21:55:151
3047499128,3cyclictest18687-21sh23:02:251
3047499127,4cyclictest1714-21ssh21:54:581
3047499125,6cyclictest22727-21ssh23:05:141
3047499125,4cyclictest29992-21aten2_r7power_v22:50:151
30474991210,1cyclictest6677-21head19:25:171
3047499120,12cyclictest0-21swapper/122:29:551
3047499120,11cyclictest19133-21tr00:20:161
3047499120,11cyclictest0-21swapper/123:49:531
3047499120,10cyclictest0-21swapper/121:34:511
3046999129,2cyclictest6030-21cat20:30:240
3046999129,2cyclictest32242-21basename23:30:150
3046999129,2cyclictest31870-21cron20:20:010
3046999129,2cyclictest28064-21grep20:10:170
3046999129,2cyclictest25187-21grep21:10:230
3046999129,2cyclictest1936-21rm23:50:200
3046999129,2cyclictest11872-21tr20:45:150
3046999128,3cyclictest18083-21grep19:50:140
3046999128,3cyclictest18083-21grep19:50:140
3046999128,3cyclictest14195-21tr19:40:200
3046999128,3cyclictest11152-21df19:35:150
30469991210,1cyclictest24273-21latency_hist23:45:000
3046999120,6cyclictest0-21swapper/020:40:140
3047999119,1cyclictest13222-21ls00:35:273
3047999118,2cyclictest8890-21sort19:30:163
3047999118,2cyclictest3686-21tr19:20:153
3047999118,2cyclictest32074-21ssh22:31:243
3047999118,2cyclictest31608-21chrt20:16:443
3047999118,2cyclictest28620-21ssh23:46:443
3047999118,2cyclictest2566-21ssh22:14:393
3047999118,2cyclictest16072-21grep19:45:163
3047999118,2cyclictest10475-21cut21:40:183
3047999118,2cyclictest10315-21ssh23:16:363
3047999117,3cyclictest1483-21ssh23:50:173
3047999117,2cyclictest22678-21tr20:00:143
3047699119,1cyclictest32133-21/usr/sbin/munin20:20:112
3047699118,2cyclictest29374-21rm23:47:192
3047499118,3cyclictest15422-21rm23:39:371
3047499118,2cyclictest5087-21ssh23:14:021
3047499118,2cyclictest5087-21ssh23:14:021
3047499118,2cyclictest4397-21ssh00:10:381
3047499118,2cyclictest4259-21grep19:20:171
3047499118,2cyclictest32199-21cut20:20:121
3047499118,2cyclictest30746-21basename19:10:131
3047499118,2cyclictest25628-21grep20:05:161
3047499118,2cyclictest19075-21ssh22:43:321
3047499118,2cyclictest15009-21cat20:50:191
3047499118,2cyclictest14251-21sed19:40:221
3047499118,2cyclictest12317-21ssh22:39:471
3047499116,4cyclictest10341-21needreboot20:40:211
3047499115,4cyclictest10404-21munin-run19:34:591
3047499113,4cyclictest0-21swapper/120:35:231
3046999118,2cyclictest9614-21tr23:55:150
3046999118,2cyclictest7762-21ssh21:39:100
3046999118,2cyclictest5651-21ssh22:55:060
3046999118,2cyclictest32433-21grep20:20:130
3046999118,2cyclictest32126-21ssh21:15:110
3046999118,2cyclictest31457-21sed19:10:160
3046999118,2cyclictest31181-21tr23:10:140
3046999118,2cyclictest31181-21tr23:10:140
3046999118,2cyclictest3033-21interrupts20:25:180
3046999118,2cyclictest3014-21ssh00:30:110
3046999118,2cyclictest28736-21ssh23:46:480
3046999118,2cyclictest27850-21grep22:10:140
3046999118,2cyclictest27404-21tr00:25:140
3046999118,2cyclictest25336-21ssh22:27:520
3046999118,2cyclictest23164-21ssh23:25:010
3046999118,2cyclictest21228-21ssh22:45:020
3046999118,2cyclictest20084-21ssh00:00:460
3046999118,2cyclictest19635-21head00:20:180
3046999118,2cyclictest19395-21cut21:00:170
3046999118,2cyclictest16636-21ssh23:00:480
3046999118,2cyclictest16374-21rm23:20:200
3046999118,2cyclictest15533-21grep19:45:130
3046999118,2cyclictest15256-21ssh22:21:320
3046999118,2cyclictest13027-21ssh21:22:420
3046999118,2cyclictest12146-21ssh22:00:230
3046999118,2cyclictest10289-21ssh00:15:060
3046999117,3cyclictest14764-21sed20:50:170
3046999110,10cyclictest3004-21tune2fs21:55:170
3047999108,1cyclictest27788-21cat20:10:153
3047999108,1cyclictest124072sleep319:35:293
3047999108,1cyclictest10217-21grep20:40:193
3047999107,2cyclictest26561-21grep20:05:273
3047999107,2cyclictest22132-21taskset19:58:103
3047999107,2cyclictest15719-21head20:50:263
3047999107,2cyclictest1054-21wc20:20:213
3047999106,1cyclictest225-21systemd-journal19:14:503
3047999103,3cyclictest225-21systemd-journal20:27:163
3047699108,1cyclictest44482sleep220:30:022
3047699108,1cyclictest2302-21tr19:15:232
3047699108,1cyclictest19606-21taskset19:51:152
3047699108,1cyclictest19606-21taskset19:51:142
3047699108,1cyclictest12165-21timerandwakeup19:35:252
3047699100,9cyclictest220452sleep219:57:042
3047499108,1cyclictest30063-21expr20:15:141
3047499108,1cyclictest2411-21cstates20:25:141
3047499108,1cyclictest22826-21grep20:00:151
3047499108,1cyclictest20616-21expr19:55:141
3047499108,1cyclictest13120-21cat20:45:261
3047499107,2cyclictest28252-21uname20:10:181
3047499100,9cyclictest10850-21grep19:35:141
3046999108,1cyclictest6861-21cut19:25:200
3046999108,1cyclictest29563-21ssh22:50:020
3046999108,1cyclictest22759-21chrt20:00:150
3046999107,2cyclictest25230-21tr20:05:140
3046999107,2cyclictest18311-21seq20:57:550
3046999100,9cyclictest5714-21cat19:24:590
304769996,2cyclictest13326-21grep19:40:142
304749996,2cyclictest5089-21cut20:30:161
304699997,1cyclictest1329-21cat19:15:150
304769983,1cyclictest141rcu_preempt19:45:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional