You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 05:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Apr 06, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
566799508,4cyclictest131rcu_preempt22:00:150
567599481,31cyclictest4387-21ssh23:05:111
567599457,24cyclictest131rcu_preempt22:05:131
5667994013,9cyclictest12-21ksoftirqd/021:05:130
5687993520,7cyclictest28366irq/58-eth022:05:143
5683993511,17cyclictest0-21swapper/221:20:252
567599352,23cyclictest8144-21ssh22:48:291
566799355,4cyclictest131rcu_preempt23:05:120
5687993324,6cyclictest0-21swapper/300:25:123
567599321,25cyclictest11592-21ssh23:45:131
567599316,16cyclictest21-21ksoftirqd/119:30:121
5667993010,13cyclictest12-21ksoftirqd/000:10:160
5687992913,11cyclictest28366irq/58-eth022:25:133
568399298,3cyclictest0-21swapper/219:15:242
5683992918,8cyclictest13416-21sh21:40:142
5683992910,3cyclictest27-21ksoftirqd/221:35:132
567599290,17cyclictest18099-21wc21:00:291
568399283,5cyclictest333-21kthreadcore21:50:202
568399283,18cyclictest18849-21ssh21:25:122
568399283,18cyclictest18849-21ssh21:25:122
5687992718,6cyclictest28366irq/58-eth021:50:133
5683992722,3cyclictest27766-21ssh23:00:112
5683992719,5cyclictest27-21ksoftirqd/222:35:132
568399271,6cyclictest816866ssh22:11:132
5675992715,3cyclictest31907-21tail21:30:291
5675992712,6cyclictest0-21swapper/123:15:111
568399263,6cyclictest131rcu_preempt22:35:012
568399261,9cyclictest0-21swapper/221:14:262
568399261,7cyclictest2805866ssh23:53:432
568399261,7cyclictest2669466ssh22:21:552
568399261,6cyclictest1091666ssh23:44:422
567599268,3cyclictest9304-21kthreadcore00:00:211
567599262,18cyclictest21-21ksoftirqd/121:35:131
567599262,16cyclictest2591-21ssh23:40:131
566799269,7cyclictest12-21ksoftirqd/021:15:190
568399251,7cyclictest510366ssh22:29:032
568399251,16cyclictest17033-21ssh23:30:122
568399251,13cyclictest764-21ssh22:45:112
5683992510,12cyclictest9436-21cron21:20:022
567599252,16cyclictest4307-21memory21:15:251
567599251,7cyclictest0-21swapper/122:32:171
567599251,4cyclictest0-21swapper/123:20:141
5675992512,7cyclictest22461-21kworker/1:100:13:461
567599251,14cyclictest20820-21find23:30:271
567599250,18cyclictest30005-21ssh21:48:481
568399244,4cyclictest131rcu_preempt00:35:132
568399242,18cyclictest27802-21ssh21:30:122
568399241,7cyclictest131rcu_preempt23:23:532
5683992410,7cyclictest11689-21memory20:20:212
567599249,9cyclictest131rcu_preempt22:29:121
567599243,4cyclictest21-21ksoftirqd/123:38:511
567599241,5cyclictest21-21ksoftirqd/121:10:121
567599241,19cyclictest2737-21ssh22:10:131
567599241,17cyclictest20756-21sh23:13:301
5675992411,7cyclictest15382-21kworker/1:121:25:371
5675992411,7cyclictest15382-21kworker/1:121:25:371
5675992411,7cyclictest13468-21kworker/1:000:05:311
567599241,12cyclictest0-21swapper/121:50:381
567599241,12cyclictest0-21swapper/100:26:591
566799241,4cyclictest131rcu_preempt22:21:400
5667992413,8cyclictest12-21ksoftirqd/019:20:120
568799233,15cyclictest0-21swapper/322:40:123
568799232,17cyclictest0-21swapper/322:45:113
5687992317,3cyclictest28366irq/58-eth023:15:113
568399232,18cyclictest24288-21ssh22:40:132
568399231,6cyclictest3162466ssh00:13:582
5683992313,8cyclictest0-21swapper/220:45:212
568399231,19cyclictest4049-21irqrtprio20:15:192
5683992311,9cyclictest12533-21ps19:50:182
568399231,10cyclictest1167866ssh22:15:132
567599237,9cyclictest131rcu_preempt00:34:061
567599231,7cyclictest21-21ksoftirqd/121:59:521
567599231,4cyclictest0-21swapper/122:00:461
567599231,3cyclictest6996-21ntpq20:15:251
567599231,3cyclictest0-21swapper/122:15:251
567599231,15cyclictest0-21swapper/122:55:021
567599231,14cyclictest0-21swapper/123:27:391
567599231,13cyclictest0-21swapper/122:22:301
567599231,11cyclictest17375-21cat21:40:281
567599231,11cyclictest10793-21tr21:20:191
566799233,3cyclictest131rcu_preempt21:55:370
566799233,3cyclictest131rcu_preempt00:34:000
566799232,4cyclictest131rcu_preempt23:55:400
566799232,4cyclictest131rcu_preempt23:55:400
566799232,4cyclictest131rcu_preempt23:46:110
566799232,4cyclictest131rcu_preempt20:30:150
566799232,15cyclictest31440-21ssh21:50:130
566799231,7cyclictest131rcu_preempt23:50:290
5667992317,3cyclictest0-21swapper/021:40:130
566799231,4cyclictest131rcu_preempt23:40:170
566799231,4cyclictest131rcu_preempt21:48:320
5667992312,3cyclictest131rcu_preempt19:25:130
568799226,11cyclictest0-21swapper/323:35:123
5687992213,6cyclictest28366irq/58-eth022:20:133
568799221,18cyclictest28366irq/58-eth022:50:123
568399228,6cyclictest9410-21cut21:55:172
568399228,11cyclictest20238-21sed23:50:012
568399227,12cyclictest27-21ksoftirqd/200:25:192
568399221,7cyclictest2052366ssh00:06:242
568399221,6cyclictest770466ssh00:18:152
568399221,6cyclictest2757466ssh23:15:502
568399221,6cyclictest1355666ssh00:03:452
568399221,5cyclictest1726466ssh00:24:012
5683992211,9cyclictest0-21swapper/222:05:142
5683992211,9cyclictest0-21swapper/219:55:222
568399221,12cyclictest22010-21cut22:55:222
5683992210,5cyclictest29595-21sshd19:32:222
567599228,10cyclictest0-21swapper/119:50:131
567599222,2cyclictest131rcu_preempt23:50:281
567599221,7cyclictest0-21swapper/122:37:341
567599221,3cyclictest0-21swapper/100:21:441
567599221,15cyclictest3051-21kworker/1:000:37:501
567599221,15cyclictest0-21swapper/122:43:571
567599221,14cyclictest3953-21kthreadcore19:40:191
567599221,12cyclictest17933-21latency_hist20:30:021
5675992210,3cyclictest0-21swapper/123:00:181
567599220,4cyclictest1603-21ssh00:15:191
566799227,4cyclictest131rcu_preempt20:40:110
566799226,8cyclictest0-21swapper/023:10:490
566799223,3cyclictest131rcu_preempt00:39:450
566799223,3cyclictest131rcu_preempt00:20:210
566799222,4cyclictest131rcu_preempt20:35:240
566799221,8cyclictest31922-21irqcore23:20:160
566799221,7cyclictest28891-21cut23:00:180
566799221,4cyclictest131rcu_preempt23:26:560
566799221,4cyclictest131rcu_preempt22:57:510
566799221,4cyclictest131rcu_preempt21:36:290
566799221,4cyclictest131rcu_preempt00:25:340
566799221,4cyclictest12-21ksoftirqd/023:32:250
5667992213,7cyclictest10010-21sshd22:13:280
5667992212,7cyclictest13566-21ntpq21:20:260
566799220,6cyclictest131rcu_preempt21:27:200
566799220,6cyclictest131rcu_preempt21:27:190
568799219,9cyclictest0-21swapper/300:18:313
568799218,9cyclictest0-21swapper/323:25:133
568799218,10cyclictest33-21ksoftirqd/321:25:173
568799218,10cyclictest33-21ksoftirqd/321:25:163
568799218,10cyclictest28366irq/58-eth023:50:133
568799213,12cyclictest0-21swapper/321:10:133
5687992112,6cyclictest28366irq/58-eth022:10:243
568399219,9cyclictest20369-21fschecks_count19:25:162
568399217,2cyclictest131rcu_preempt19:35:272
568399217,11cyclictest11789-21kthreadcore22:50:212
568399216,2cyclictest131rcu_preempt19:10:192
568399216,2cyclictest131rcu_preempt19:10:192
568399212,7cyclictest131rcu_preempt22:00:172
5683992111,8cyclictest3592-21latency_hist00:35:012
5683992111,8cyclictest0-21swapper/220:10:292
568399211,17cyclictest16117-21kthreadcore21:00:212
568399211,12cyclictest965-21sort23:55:282
568399211,12cyclictest965-21sort23:55:282
568399211,11cyclictest30380-21usage_Anybus23:35:302
568399211,10cyclictest8064-21sh23:25:142
568399211,10cyclictest27359-21munin-run20:40:002
5683992110,4cyclictest27-21ksoftirqd/223:10:022
567599215,8cyclictest18681-21tr19:55:191
567599211,9cyclictest0-21swapper/120:50:191
567599211,3cyclictest0-21swapper/120:45:261
567599211,12cyclictest9052-21grep20:20:181
567599211,12cyclictest1924-21cat23:55:311
567599211,12cyclictest1924-21cat23:55:311
567599211,11cyclictest26028-21munin-run20:05:021
567599211,11cyclictest0-21swapper/121:05:261
567599210,7cyclictest171-21systemd-network22:53:271
566799216,8cyclictest12-21ksoftirqd/019:15:130
566799216,3cyclictest131rcu_preempt19:40:110
566799215,3cyclictest131rcu_preempt20:00:120
566799214,4cyclictest131rcu_preempt23:40:020
566799213,2cyclictest131rcu_preempt19:30:170
566799212,3cyclictest131rcu_preempt21:12:240
566799211,4cyclictest131rcu_preempt22:40:300
566799211,4cyclictest131rcu_preempt22:31:410
566799211,3cyclictest131rcu_preempt00:19:160
566799211,2cyclictest131rcu_preempt20:15:310
5667992110,3cyclictest12-21ksoftirqd/022:15:290
5667992110,2cyclictest12-21ksoftirqd/022:10:010
568799209,9cyclictest0-21swapper/300:38:363
568799204,13cyclictest0-21swapper/300:20:123
568799204,10cyclictest0-21swapper/321:40:123
568799203,13cyclictest0-21swapper/323:30:123
5687992013,5cyclictest0-21swapper/323:10:123
5687992012,5cyclictest0-21swapper/300:00:143
5687992012,4cyclictest28366irq/58-eth023:40:133
5687992011,6cyclictest28366irq/58-eth021:46:173
568799201,12cyclictest28366irq/58-eth023:55:153
568799201,12cyclictest28366irq/58-eth023:55:143
568399209,8cyclictest22758-21sh21:45:182
568399209,4cyclictest27-21ksoftirqd/220:50:122
568399207,10cyclictest0-21swapper/219:45:162
568399202,10cyclictest18740-21cat19:20:232
568399201,11cyclictest3190-21kthreadcore19:40:182
568399201,11cyclictest13858-21ssh23:10:182
5683992010,7cyclictest27-21ksoftirqd/221:10:022
5683992010,7cyclictest27-21ksoftirqd/220:40:272
567599205,8cyclictest633-21Async12:11:531
567599203,5cyclictest21-21ksoftirqd/119:15:211
567599201,12cyclictest0-21swapper/120:30:141
567599201,11cyclictest8984-21kthreadcore19:45:201
567599201,11cyclictest30972-21ls20:10:011
567599201,11cyclictest20101-21latency_hist19:25:021
567599201,10cyclictest0-21swapper/119:30:021
567599200,10cyclictest481-21tr19:35:211
566799204,3cyclictest131rcu_preempt00:05:290
566799203,3cyclictest131rcu_preempt21:00:010
566799202,2cyclictest131rcu_preempt22:48:000
566799202,2cyclictest131rcu_preempt20:50:120
566799201,4cyclictest131rcu_preempt19:45:290
566799201,3cyclictest131rcu_preempt22:25:040
566799201,16cyclictest6050-21ssh00:00:130
5667992010,3cyclictest12-21ksoftirqd/020:20:120
568799199,7cyclictest0-21swapper/322:31:293
568799199,7cyclictest0-21swapper/321:20:233
568799193,11cyclictest0-21swapper/322:35:143
568799193,11cyclictest0-21swapper/321:55:123
568799193,10cyclictest0-21swapper/322:55:123
568799192,12cyclictest0-21swapper/323:00:123
5687991913,5cyclictest321rcuc/300:10:173
5687991911,5cyclictest0-21swapper/300:30:283
568399191,10cyclictest9796-21fschecks_count20:55:152
568399191,10cyclictest26673-21irqcore20:05:182
568399191,10cyclictest21996-21irqrtprio20:00:172
568399191,10cyclictest18739-21kthreadcore20:30:182
568399191,10cyclictest13576-21irqcore20:25:182
567599197,2cyclictest131rcu_preempt20:40:261
567599191,10cyclictest3057-21cat20:10:311
566799193,3cyclictest131rcu_preempt21:32:550
566799193,2cyclictest131rcu_preempt22:35:310
566799192,2cyclictest131rcu_preempt20:50:020
5667991914,3cyclictest151rcuc/022:50:110
566799191,2cyclictest131rcu_preempt23:16:190
566799191,2cyclictest131rcu_preempt20:25:220
566799191,2cyclictest131rcu_preempt20:10:240
568799189,4cyclictest0-21swapper/323:05:283
568799181,13cyclictest0-21swapper/322:00:193
568799181,12cyclictest0-21swapper/320:05:273
5687991810,5cyclictest28366irq/58-eth023:45:143
5687991810,3cyclictest0-21swapper/321:31:423
567599181,9cyclictest0-21swapper/120:55:131
567599181,11cyclictest0-21swapper/119:10:111
567599181,11cyclictest0-21swapper/119:10:101
566799188,7cyclictest29927-21df_inode19:35:150
566799187,3cyclictest12-21ksoftirqd/019:50:130
566799181,2cyclictest131rcu_preempt21:00:130
566799181,2cyclictest131rcu_preempt20:05:300
566799181,2cyclictest131rcu_preempt19:55:250
566799181,2cyclictest131rcu_preempt19:10:180
566799181,2cyclictest131rcu_preempt19:10:180
568799179,6cyclictest0-21swapper/322:16:023
568799179,5cyclictest28366irq/58-eth021:15:153
568799179,5cyclictest0-21swapper/300:05:303
568799179,3cyclictest0-21swapper/319:30:243
568799171,2cyclictest0-21swapper/321:05:123
5687991711,4cyclictest0-21swapper/323:20:133
5687991711,4cyclictest0-21swapper/321:35:163
568399161,12cyclictest5792-21latency_hist19:10:012
568799152,10cyclictest0-21swapper/320:45:053
568799152,10cyclictest0-21swapper/320:00:133
568799152,10cyclictest0-21swapper/319:20:133
568799151,11cyclictest0-21swapper/319:55:183
567599151,4cyclictest5775-21latency_hist19:10:011
568799149,3cyclictest0-21swapper/320:44:453
568799144,5cyclictest0-21swapper/319:25:143
568799142,9cyclictest0-21swapper/320:55:123
568799142,9cyclictest0-21swapper/319:10:123
568799142,9cyclictest0-21swapper/319:10:123
568799142,8cyclictest0-21swapper/319:45:133
568799141,7cyclictest0-21swapper/320:30:013
566799141,2cyclictest131rcu_preempt19:10:010
568799137,3cyclictest0-21swapper/319:40:133
568799133,7cyclictest0-21swapper/320:30:133
568799132,8cyclictest0-21swapper/320:35:173
568799132,8cyclictest0-21swapper/319:40:023
568799132,8cyclictest0-21swapper/319:15:133
568799131,9cyclictest0-21swapper/321:05:023
568799131,9cyclictest0-21swapper/320:20:273
568799131,6cyclictest0-21swapper/320:15:283
568799131,6cyclictest0-21swapper/320:10:183
568799131,10cyclictest0-21swapper/320:50:293
568799121,8cyclictest0-21swapper/319:50:163
568799111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional