You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 01:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Thu Apr 02, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
158929866,8sleep20-21swapper/207:07:482
142629077,9sleep00-21swapper/007:05:400
159728157,6sleep30-21swapper/307:07:533
145627958,6sleep10-21swapper/107:06:031
43322640,0sleep30-21swapper/310:58:003
41942460,0sleep30-21swapper/309:40:183
1858993029,1cyclictest0-21swapper/209:53:352
1861992810,7cyclictest141rcu_preempt11:36:313
185899270,27cyclictest0-21swapper/212:07:352
185899260,26cyclictest0-21swapper/209:31:502
185699260,21cyclictest141rcu_preempt11:46:181
185699256,8cyclictest25-21ksoftirqd/112:20:171
1856992521,4cyclictest25-21ksoftirqd/109:10:001
1858992421,2cyclictest32-21ksoftirqd/211:40:002
185899240,23cyclictest0-21swapper/209:44:212
185899240,1cyclictest0-21swapper/209:59:012
1856992422,2cyclictest25-21ksoftirqd/111:25:441
185199240,1cyclictest0-21swapper/010:10:140
1858992323,0cyclictest0-21swapper/210:45:362
185899230,23cyclictest0-21swapper/209:27:192
185699230,1cyclictest0-21swapper/111:08:261
186199227,14cyclictest39-21ksoftirqd/311:01:203
185899226,15cyclictest0-21swapper/211:05:382
1858992221,1cyclictest0-21swapper/209:12:492
185899220,22cyclictest0-21swapper/212:21:422
185899220,22cyclictest0-21swapper/211:33:512
185899220,22cyclictest0-21swapper/210:19:592
185899220,22cyclictest0-21swapper/209:47:022
185699221,21cyclictest25-21ksoftirqd/110:05:401
1851992221,1cyclictest0-21swapper/011:45:390
1861992119,1cyclictest17045-21diskmemload12:03:233
1861992117,3cyclictest0-21swapper/312:30:023
1861992110,1cyclictest141rcu_preempt07:25:133
185899210,21cyclictest0-21swapper/210:39:252
185699213,2cyclictest0-21swapper/110:32:411
1856992120,1cyclictest0-21swapper/111:16:551
1856992119,1cyclictest0-21swapper/110:26:011
1856992117,4cyclictest38150irq/131-snd_hda11:42:151
185699210,20cyclictest0-21swapper/109:00:191
185199210,20cyclictest0-21swapper/011:58:300
1861992019,1cyclictest39-21ksoftirqd/310:46:373
1861992019,1cyclictest17045-21diskmemload11:22:593
1861992016,4cyclictest0-21swapper/309:15:003
1861992015,4cyclictest29522-21grep09:55:183
186199200,1cyclictest0-21swapper/311:52:133
185899200,20cyclictest0-21swapper/212:19:182
185899200,19cyclictest0-21swapper/210:50:572
185899200,19cyclictest0-21swapper/210:50:562
185199200,19cyclictest0-21swapper/010:56:100
186199197,2cyclictest26835-21sh12:29:203
186199195,12cyclictest0-21swapper/311:10:163
1861991912,7cyclictest0-21swapper/310:40:123
1861991910,8cyclictest17045-21diskmemload11:41:063
186199190,4cyclictest0-21swapper/312:38:143
186199190,4cyclictest0-21swapper/312:38:143
185899192,1cyclictest141rcu_preempt10:44:442
1856991915,1cyclictest25-21ksoftirqd/111:12:291
185699190,4cyclictest141rcu_preempt11:54:001
185699190,1cyclictest0-21swapper/110:45:281
1851991919,0cyclictest0-21swapper/010:29:170
185199190,18cyclictest0-21swapper/012:25:060
186199187,6cyclictest381rcuc/312:15:153
1861991817,1cyclictest0-21swapper/312:20:213
1861991816,1cyclictest17045-21diskmemload09:17:373
186199181,4cyclictest0-21swapper/310:24:243
186199180,1cyclictest0-21swapper/311:57:123
185899180,18cyclictest0-21swapper/211:22:172
185899180,18cyclictest0-21swapper/210:09:432
185699185,13cyclictest25-21ksoftirqd/110:51:081
185699185,13cyclictest25-21ksoftirqd/110:51:071
1856991816,2cyclictest25-21ksoftirqd/110:03:161
1856991812,6cyclictest25-21ksoftirqd/112:10:121
185699180,1cyclictest141rcu_preempt10:43:031
186199174,12cyclictest0-21swapper/310:15:143
186199173,10cyclictest0-21swapper/310:35:123
1861991713,4cyclictest39-21ksoftirqd/311:17:353
1861991710,6cyclictest0-21swapper/309:24:203
1858991712,3cyclictest32-21ksoftirqd/209:15:012
185899170,17cyclictest0-21swapper/212:04:432
1856991715,1cyclictest241rcuc/112:30:251
185699171,1cyclictest141rcu_preempt09:47:001
185199174,0cyclictest141rcu_preempt10:00:320
1851991717,0cyclictest0-21swapper/009:45:100
1851991716,1cyclictest0-21swapper/010:09:260
185199170,2cyclictest0-21swapper/011:41:530
185199170,17cyclictest0-21swapper/012:21:000
186199167,7cyclictest39-21ksoftirqd/310:29:183
186199167,7cyclictest17045-21diskmemload11:08:083
186199164,11cyclictest23698-21sensors10:30:213
1861991614,2cyclictest39-21ksoftirqd/310:07:393
1861991614,1cyclictest0-21swapper/311:25:143
185899160,15cyclictest0-21swapper/211:15:182
185699168,4cyclictest17045-21diskmemload12:39:171
185699168,4cyclictest17045-21diskmemload12:39:171
185699162,2cyclictest25-21ksoftirqd/110:12:071
1856991615,0cyclictest0-21swapper/108:40:191
1856991614,2cyclictest241rcuc/109:10:151
1856991613,3cyclictest25-21ksoftirqd/110:35:131
185699160,14cyclictest0-21swapper/110:16:141
185199161,3cyclictest0-21swapper/007:25:100
186199157,7cyclictest0-21swapper/310:01:373
1861991514,1cyclictest39-21ksoftirqd/310:53:513
1861991514,1cyclictest39-21ksoftirqd/310:53:513
1861991512,2cyclictest0-21swapper/311:32:213
1861991511,3cyclictest0-21swapper/309:31:243
1861991510,1cyclictest0-21swapper/311:47:023
186199150,2cyclictest0-21swapper/309:45:023
185899159,5cyclictest13015-21cut09:00:162
1858991513,2cyclictest27272-21kworker/u8:207:40:032
185899151,0cyclictest141rcu_preempt12:25:462
185899150,15cyclictest0-21swapper/209:35:492
185899150,13cyclictest0-21swapper/210:31:052
185699158,1cyclictest141rcu_preempt12:09:461
185699154,4cyclictest25-21ksoftirqd/109:59:521
185699152,6cyclictest25-21ksoftirqd/109:23:131
1856991514,1cyclictest0-21swapper/110:24:241
1856991510,5cyclictest25-21ksoftirqd/112:25:121
185699150,1cyclictest141rcu_preempt09:51:231
1851991515,0cyclictest0-21swapper/012:36:420
1851991515,0cyclictest0-21swapper/012:36:410
185199150,1cyclictest0-21swapper/012:17:360
185199150,1cyclictest0-21swapper/011:08:040
185199150,15cyclictest0-21swapper/009:56:160
185199150,0cyclictest0-21swapper/011:01:020
1861991412,1cyclictest0-21swapper/312:11:113
1861991410,3cyclictest17045-21diskmemload12:06:533
1861991410,3cyclictest0-21swapper/309:37:563
186199140,2cyclictest0-21swapper/309:50:463
185899149,3cyclictest10118-21wc12:00:002
185899149,1cyclictest25670-21munin-node10:00:122
1858991410,3cyclictest29533-21head08:25:262
185899140,13cyclictest0-21swapper/211:01:072
185899140,0cyclictest141rcu_preempt11:10:162
185899140,0cyclictest0-21swapper/210:22:262
185699147,7cyclictest17045-21diskmemload11:31:301
185699146,5cyclictest0-21swapper/109:27:331
185699143,0cyclictest16534-21kworker/1:211:59:491
185699142,11cyclictest0-21swapper/110:56:221
1856991413,1cyclictest0-21swapper/111:39:131
1856991412,2cyclictest25-21ksoftirqd/111:20:131
185699141,0cyclictest0-21swapper/109:44:051
185699140,1cyclictest0-21swapper/109:31:501
1851991413,1cyclictest0-21swapper/012:15:000
1851991410,3cyclictest9567-21cron09:25:000
185199140,8cyclictest14210-21perf07:50:000
186199133,7cyclictest0-21swapper/310:10:473
186199132,4cyclictest225-21systemd-journal08:17:583
1861991311,1cyclictest17045-21diskmemload09:25:123
1861991310,2cyclictest18369-21grep08:05:093
185899139,3cyclictest25730-21tr08:20:142
185899131,1cyclictest141rcu_preempt10:59:522
1858991311,1cyclictest8845-21apache209:05:022
1858991310,2cyclictest23178-21basename08:15:122
1858991310,2cyclictest21418-21rm10:10:142
1858991310,2cyclictest14067-21tr10:25:142
185899130,3cyclictest10966-21apt07:40:002
185899130,13cyclictest0-21swapper/212:13:512
185699138,3cyclictest241rcuc/111:05:001
185699137,5cyclictest25-21ksoftirqd/112:00:161
185699137,5cyclictest25-21ksoftirqd/107:10:161
185699133,2cyclictest477-21rs:main1
1856991310,2cyclictest26710-21wc08:20:211
185199139,3cyclictest17913-21cron09:30:000
1851991310,2cyclictest9750-21wc08:55:000
1851991310,2cyclictest8923-21cut07:35:090
1851991310,2cyclictest24419-21ssh12:08:170
186199129,2cyclictest26538-21cut08:20:183
186199129,2cyclictest17206-21awk08:00:013
186199128,3cyclictest39-21ksoftirqd/307:19:043
186199127,1cyclictest9676-21ls08:55:003
186199125,6cyclictest25670-21munin-node08:48:413
186199123,3cyclictest225-21systemd-journal09:04:593
186199120,11cyclictest14825-21awk07:55:013
185899129,2cyclictest8234-21grep08:50:152
185899129,2cyclictest20430-21awk08:10:012
185899128,3cyclictest3796-21wc08:40:192
185899128,3cyclictest17979-21wc11:25:192
185899124,4cyclictest8524-21cron07:35:002
185899123,4cyclictest225-21systemd-journal09:23:162
1858991211,1cyclictest32-21ksoftirqd/212:32:022
1858991210,1cyclictest6713-21wc08:45:252
185699129,2cyclictest4915-21sleep07:19:591
185699128,3cyclictest25-21ksoftirqd/109:40:011
185699124,2cyclictest25-21ksoftirqd/109:15:241
1856991210,1cyclictest1796-21grep08:35:251
1856991210,1cyclictest11838-21basename07:45:001
185199129,2cyclictest5611-21ssh10:39:490
185199129,2cyclictest3840-21grep07:15:150
185199129,2cyclictest30907-21grep08:30:170
185199129,2cyclictest27308-21ssh09:15:220
185199129,2cyclictest24262-21ssh11:29:440
185199129,2cyclictest23955-21head10:50:170
185199129,2cyclictest23955-21head10:50:170
185199129,2cyclictest22187-21cat08:10:250
185199129,2cyclictest20426-21munin-run08:10:010
185199129,2cyclictest1578-21wc11:55:000
185199129,2cyclictest10459-21ssh12:00:130
185199125,6cyclictest24080-21ssh11:10:110
1851991211,1cyclictest0-21swapper/012:32:120
1851991211,1cyclictest0-21swapper/011:16:040
185199120,1cyclictest141rcu_preempt10:21:090
186199118,2cyclictest8573-21cat07:35:003
186199118,2cyclictest2997-21tr08:40:143
186199118,2cyclictest2229-21chrt07:11:583
186199118,2cyclictest16519-21cat09:05:263
186199118,2cyclictest12133-21cut07:45:103
186199118,2cyclictest12002-21wc07:45:003
186199118,2cyclictest10174-21grep08:55:143
186199111,5cyclictest20683-21apt-get08:10:143
185899118,2cyclictest9246-21ssh12:37:402
185899118,2cyclictest9246-21ssh12:37:402
185899118,2cyclictest5509-21taskset07:24:482
185899118,2cyclictest4283-21wc07:15:202
185899118,2cyclictest32132-21taskset08:32:582
185899118,2cyclictest21439-21grep08:10:172
185899118,2cyclictest17825-21cut11:45:132
185899118,2cyclictest16291-21head07:55:162
185899118,2cyclictest14802-21cron07:55:012
185899118,2cyclictest12434-21expr07:45:112
185899118,2cyclictest12406-21ssh11:41:112
185899117,3cyclictest26706-21ssh11:50:142
185699119,2cyclictest32139-21taskset08:33:021
185699119,1cyclictest30121-21sort08:30:001
185699118,2cyclictest13174-21cat07:45:151
185699116,2cyclictest225-21systemd-journal07:22:521
185699113,7cyclictest241rcuc/107:25:091
185699113,4cyclictest25-21ksoftirqd/112:15:241
185199119,1cyclictest29526-21sed08:25:250
185199119,1cyclictest0-21swapper/010:30:120
185199118,2cyclictest7065-21sort10:40:140
185199118,2cyclictest627-21grep08:35:150
185199118,2cyclictest4548-21ssh11:36:370
185199118,2cyclictest3746-21head09:40:160
185199118,2cyclictest31362-21ssh09:37:290
185199118,2cyclictest29235-21ssh11:32:060
185199118,2cyclictest2555-21ssh10:18:120
185199118,2cyclictest23077-21ssh09:51:310
185199118,2cyclictest20212-21ssh09:10:580
185199118,2cyclictest18740-21wc09:30:130
185199118,2cyclictest18013-21cut08:05:010
185199118,2cyclictest15254-21expr10:45:140
185199118,2cyclictest11744-21chrt08:56:080
185199115,5cyclictest4890-21munin-run08:45:000
185199114,5cyclictest9013-21df_abs11:20:140
185199111,5cyclictest17065-21perf09:09:590
186199107,2cyclictest32564-21apache_accesses08:35:103
186199107,2cyclictest30690-21sed08:30:153
186199107,2cyclictest28069-21grep08:25:143
186199107,2cyclictest10236-21sensors07:35:173
185899108,1cyclictest2134-21chrt08:38:262
185899108,1cyclictest17555-21chrt08:01:412
185899108,1cyclictest0-21swapper/207:25:122
185899107,2cyclictest11611-21tail08:55:252
185899100,5cyclictest225-21systemd-journal07:10:162
185699109,1cyclictest17474-21taskset08:00:381
185699108,2cyclictest7158-21chrt08:49:001
185699108,2cyclictest18697-21cat08:05:101
185699108,1cyclictest9239-21sed08:50:251
185699107,2cyclictest11033-21grep08:55:191
185699100,10cyclictest9283-21expr07:35:101
185699100,10cyclictest0-21swapper/107:55:061
185199108,2cyclictest13455-21awk09:00:200
185199108,1cyclictest8277-21chrt07:31:490
185199108,1cyclictest5462-21expr08:45:140
185199108,1cyclictest26747-21processes08:20:210
185199108,1cyclictest2110-21chrt07:10:260
185199108,1cyclictest15452-21expr07:55:100
185199108,1cyclictest14581-21taskset07:52:120
18619996,2cyclictest5459-21chrt07:24:083
18619996,2cyclictest17636-21chrt08:02:423
18569998,1cyclictest83752sleep107:33:061
18569998,1cyclictest14612-21seq07:52:371
18569997,1cyclictest22363-21sed08:10:251
18519997,1cyclictest5185-21taskset07:20:350
18519997,1cyclictest24441-21sensors08:15:220
18519997,1cyclictest11574-21chrt07:42:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional