You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-22 - 07:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Fri May 22, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1390991070,84rtkit-daemon0-21swapper/019:08:290
288687728343,31sleep30-21swapper/319:05:263
288647527444,21sleep20-21swapper/219:05:122
288784026747,14sleep10-21swapper/119:09:011
139099560,4rtkit-daemon1389-21rtkit-daemon00:31:090
139099520,4rtkit-daemon1389-21rtkit-daemon20:45:123
139099510,4rtkit-daemon1389-21rtkit-daemon20:15:113
139099480,3rtkit-daemon1389-21rtkit-daemon19:34:273
139099440,3rtkit-daemon1389-21rtkit-daemon23:21:403
139099420,3rtkit-daemon1389-21rtkit-daemon23:03:553
30044572400,3sleep13056-21PK-Backend00:35:131
288806199387,4cyclictest34-21ksoftirqd/200:00:012
29200952360,9sleep342-21ksoftirqd/320:40:003
2888057993311,11cyclictest26-21ksoftirqd/100:10:001
139099320,5rtkit-daemon1788643-21eadedCompositor20:08:293
139099320,4rtkit-daemon1788643-21eadedCompositor19:57:333
2888061993116,4cyclictest34-21ksoftirqd/221:55:012
2888061993010,10cyclictest34-21ksoftirqd/221:45:002
139099300,2rtkit-daemon0-21swapper/321:35:543
288806199295,3cyclictest331rcuc/200:15:032
2888061992924,4cyclictest34-21ksoftirqd/222:45:012
288806199291,5cyclictest2921960-21/usr/sbin/munin20:45:152
2888061992913,5cyclictest34-21ksoftirqd/223:30:012
288805799292,3cyclictest26-21ksoftirqd/123:30:011
288805499292,20cyclictest0-21swapper/020:15:130
288806199285,12cyclictest34-21ksoftirqd/223:40:012
2888061992814,5cyclictest34-21ksoftirqd/220:40:012
288806199280,3cyclictest2989253-21gpgconf23:55:012
288805799282,4cyclictest26-21ksoftirqd/121:45:001
2888063992718,4cyclictest42-21ksoftirqd/300:25:013
288806199274,2cyclictest141rcu_preempt22:00:012
2888061992716,4cyclictest34-21ksoftirqd/219:50:002
288805799275,10cyclictest26-21ksoftirqd/120:00:001
288805799272,7cyclictest26-21ksoftirqd/123:55:011
288805799272,10cyclictest26-21ksoftirqd/119:20:001
288805499278,8cyclictest13-21ksoftirqd/021:45:020
288805499272,23cyclictest2990939-40mandb00:00:010
288806399260,2cyclictest141rcu_preempt19:20:283
288806199266,10cyclictest34-21ksoftirqd/222:05:202
288805799266,9cyclictest26-21ksoftirqd/100:10:011
288805799266,9cyclictest26-21ksoftirqd/100:10:011
288805799263,8cyclictest26-21ksoftirqd/120:25:121
2888054992610,9cyclictest13-21ksoftirqd/019:55:010
288806399257,10cyclictest42-21ksoftirqd/319:50:173
288806199254,6cyclictest34-21ksoftirqd/219:14:592
288806199253,2cyclictest141rcu_preempt00:15:002
288806199253,2cyclictest141rcu_preempt00:15:002
2888061992513,10cyclictest2959655-21irqstats22:30:162
2888061992511,4cyclictest34-21ksoftirqd/222:39:082
288806199250,7cyclictest141rcu_preempt20:15:112
2888057992512,3cyclictest26-21ksoftirqd/122:45:241
288805799250,6cyclictest3056-21PK-Backend20:40:001
2888054992522,2cyclictest161rcuc/000:38:270
2888054992510,4cyclictest13-21ksoftirqd/019:20:000
288806199249,4cyclictest34-21ksoftirqd/200:39:322
288806199247,3cyclictest141rcu_preempt20:25:122
288806199242,2cyclictest141rcu_preempt23:00:012
2888061992413,8cyclictest1-21systemd23:50:002
2888061992411,4cyclictest34-21ksoftirqd/220:45:002
2888061992410,4cyclictest34-21ksoftirqd/221:39:082
2888061992410,3cyclictest34-21ksoftirqd/222:25:022
2888061992410,3cyclictest34-21ksoftirqd/200:25:212
288806199240,22cyclictest1389-21rtkit-daemon23:21:402
288805799247,4cyclictest26-21ksoftirqd/123:00:011
288805799245,7cyclictest26-21ksoftirqd/123:40:181
2888057992420,3cyclictest251rcuc/100:28:431
288805799241,3cyclictest26-21ksoftirqd/122:05:201
288805799240,7cyclictest26-21ksoftirqd/100:25:001
288805799240,6cyclictest3056-21PK-Backend19:35:001
288805799240,4cyclictest26-21ksoftirqd/119:55:021
288805499242,4cyclictest13-21ksoftirqd/021:20:000
29237172230,10sleep234-21ksoftirqd/220:50:102
288806399238,13cyclictest2961403-21kernelversion22:35:173
288806399231,1cyclictest141rcu_preempt21:35:003
2888063992310,3cyclictest42-21ksoftirqd/300:10:003
288806399230,12cyclictest2999498-21kworker/u8:000:25:013
288806199239,4cyclictest34-21ksoftirqd/221:15:212
288806199238,3cyclictest34-21ksoftirqd/222:10:202
288806199236,3cyclictest34-21ksoftirqd/222:45:152
288806199235,5cyclictest2984113-21tr23:40:122
288806199235,3cyclictest141rcu_preempt00:25:012
288806199235,2cyclictest141rcu_preempt19:20:012
288806199234,2cyclictest141rcu_preempt21:25:002
288806199233,16cyclictest2991961-21runrttasks00:00:202
288806199231,4cyclictest34-21ksoftirqd/222:00:192
2888061992311,4cyclictest34-21ksoftirqd/219:35:112
288805799234,7cyclictest26-21ksoftirqd/121:30:001
2888057992320,2cyclictest251rcuc/123:07:571
2888057992319,3cyclictest251rcuc/100:20:001
288805799231,7cyclictest26-21ksoftirqd/120:47:191
288805799231,3cyclictest26-21ksoftirqd/122:45:001
2888057992310,4cyclictest26-21ksoftirqd/121:35:471
288805799230,6cyclictest3056-21PK-Backend22:34:461
288805499239,3cyclictest141rcu_preempt22:20:000
288805499239,3cyclictest141rcu_preempt22:20:000
288805499239,2cyclictest141rcu_preempt21:00:210
288805499235,3cyclictest13-21ksoftirqd/021:52:550
288805499233,7cyclictest13-21ksoftirqd/022:45:000
288805499233,6cyclictest13-21ksoftirqd/023:10:140
288805499233,6cyclictest13-21ksoftirqd/000:25:010
288805499232,5cyclictest13-21ksoftirqd/000:10:210
288805499232,5cyclictest13-21ksoftirqd/000:10:210
288805499232,3cyclictest13-21ksoftirqd/022:35:000
288805499231,5cyclictest13-21ksoftirqd/021:45:110
2888054992314,7cyclictest2919592-21users20:35:200
2888054992314,3cyclictest13-21ksoftirqd/000:20:000
139099230,3rtkit-daemon1389-21rtkit-daemon19:40:232
2888063992216,4cyclictest2966976-21python322:50:203
288806399221,14cyclictest42-21ksoftirqd/319:35:213
2888063992210,3cyclictest42-21ksoftirqd/323:55:013
2888063992210,3cyclictest42-21ksoftirqd/322:20:013
2888063992210,3cyclictest42-21ksoftirqd/322:20:003
288806199229,3cyclictest34-21ksoftirqd/219:51:572
288806199226,9cyclictest34-21ksoftirqd/220:05:202
288806199226,8cyclictest2937578-21munin-run21:30:002
288806199225,4cyclictest34-21ksoftirqd/221:10:192
288806199222,9cyclictest34-21ksoftirqd/223:35:002
2888061992212,7cyclictest436-20systemd-journal21:00:012
2888061992212,4cyclictest34-21ksoftirqd/222:15:112
2888061992212,4cyclictest34-21ksoftirqd/222:15:112
288806199220,8cyclictest894-21irqbalance21:35:002
288806199220,8cyclictest2914790-21perf20:25:002
288806199220,8cyclictest2914790-21perf20:25:002
288806199220,21cyclictest1389-21rtkit-daemon19:34:272
288806199220,20cyclictest1389-21rtkit-daemon00:31:092
288805799227,7cyclictest2976384-21taskset23:18:381
288805799225,8cyclictest26-21ksoftirqd/121:20:001
288805799222,2cyclictest141rcu_preempt20:45:011
2888057992218,3cyclictest251rcuc/122:10:011
288805799221,10cyclictest26-21ksoftirqd/119:40:231
288805799220,8cyclictest26-21ksoftirqd/121:50:011
288805799220,7cyclictest3056-21PK-Backend21:31:031
288805499225,4cyclictest13-21ksoftirqd/019:26:280
288805499224,8cyclictest13-21ksoftirqd/020:45:110
288805499224,7cyclictest13-21ksoftirqd/023:45:210
288805499224,6cyclictest13-21ksoftirqd/019:40:100
288805499224,4cyclictest13-21ksoftirqd/021:37:490
288805499224,4cyclictest13-21ksoftirqd/021:23:450
288805499224,3cyclictest13-21ksoftirqd/019:35:430
288805499223,6cyclictest13-21ksoftirqd/022:00:010
288805499222,7cyclictest13-21ksoftirqd/020:30:000
288805499222,10cyclictest13-21ksoftirqd/022:25:020
2888054992220,1cyclictest0-21swapper/023:05:400
2888054992220,1cyclictest0-21swapper/021:07:380
2888054992218,3cyclictest161rcuc/020:05:100
288805499221,6cyclictest2956921-21munin-run22:25:000
288805499221,6cyclictest0-21swapper/023:35:160
2888054992215,5cyclictest2994374-21apt-get00:09:590
2888054992215,5cyclictest2969311-21perf23:00:000
288805499221,20cyclictest29710402sleep023:01:490
288805499220,21cyclictest3056-21PK-Backend23:17:030
288805499220,21cyclictest0-21swapper/021:32:390
288805499220,21cyclictest0-21swapper/020:55:490
139099220,4rtkit-daemon1389-21rtkit-daemon20:57:313
288806399215,14cyclictest2947509-21tr21:55:203
2888063992118,2cyclictest0-21swapper/322:04:343
2888063992115,4cyclictest2899817-21cut19:40:163
2888063992114,6cyclictest2979817-21sshd23:28:083
2888063992112,7cyclictest2987204-21unin-run23:50:003
2888063992111,4cyclictest2964143-21systemd-run22:45:003
2888063992111,3cyclictest42-21ksoftirqd/321:30:003
288806399211,12cyclictest141rcu_preempt21:10:203
2888063992110,9cyclictest42-21ksoftirqd/300:10:123
2888063992110,9cyclictest42-21ksoftirqd/300:10:123
288806399210,20cyclictest3056-21PK-Backend00:36:433
288806399210,10cyclictest0-21swapper/321:20:003
288806199218,4cyclictest34-21ksoftirqd/219:25:222
288806199216,6cyclictest141rcu_preempt20:10:102
288806199216,4cyclictest34-21ksoftirqd/220:03:482
288806199214,3cyclictest34-21ksoftirqd/222:20:112
288806199212,9cyclictest34-21ksoftirqd/220:00:002
288806199211,6cyclictest34-21ksoftirqd/223:15:002
2888061992115,4cyclictest34-21ksoftirqd/221:46:312
288806199210,9cyclictest34-21ksoftirqd/223:10:002
288806199210,20cyclictest1389-21rtkit-daemon23:03:552
288805799216,3cyclictest26-21ksoftirqd/121:50:071
288805799212,2cyclictest141rcu_preempt19:50:001
288805799211,7cyclictest251rcuc/120:07:101
2888057992112,7cyclictest2970914-21users23:00:221
288805799210,7cyclictest3056-21PK-Backend00:33:431
288805799210,6cyclictest3056-21PK-Backend23:24:501
288805799210,6cyclictest3056-21PK-Backend20:17:251
288805799210,20cyclictest3056-21PK-Backend21:04:101
288805799210,19cyclictest3056-21PK-Backend22:00:411
288805799210,19cyclictest3056-21PK-Backend20:30:501
288805799210,19cyclictest3056-21PK-Backend19:21:001
288805499219,2cyclictest141rcu_preempt20:42:420
288805499217,2cyclictest141rcu_preempt23:30:180
288805499216,2cyclictest141rcu_preempt22:35:200
288805499215,3cyclictest13-21ksoftirqd/022:11:100
288805499214,4cyclictest13-21ksoftirqd/019:23:150
288805499214,4cyclictest13-21ksoftirqd/000:01:480
288805499214,3cyclictest13-21ksoftirqd/000:25:580
288805499213,6cyclictest13-21ksoftirqd/022:50:210
288805499212,6cyclictest13-21ksoftirqd/020:20:120
288805499212,6cyclictest13-21ksoftirqd/020:20:110
288805499212,5cyclictest13-21ksoftirqd/023:40:120
2888054992119,1cyclictest0-21swapper/022:07:000
2888054992119,1cyclictest0-21swapper/020:30:070
2888054992119,1cyclictest0-21swapper/019:57:290
2888054992119,1cyclictest0-21swapper/019:46:400
2888054992118,2cyclictest161rcuc/019:13:500
288805499211,5cyclictest13-21ksoftirqd/020:10:160
288805499210,9cyclictest2964396-21cpu22:45:110
288805499210,6cyclictest13-21ksoftirqd/023:50:220
288805499210,20cyclictest0-21swapper/021:12:330
29865712200,5chrt3056-21PK-Backend23:45:201
29519832200,5chrt0-21swapper/322:10:113
288806399208,10cyclictest42-21ksoftirqd/323:30:193
288806399204,4cyclictest42-21ksoftirqd/300:00:013
2888063992018,1cyclictest3056-21PK-Backend22:59:023
2888063992012,6cyclictest436-20systemd-journal21:50:003
288806399200,1cyclictest3056-21PK-Backend19:15:013
288806399200,19cyclictest0-21swapper/323:18:563
288806399200,19cyclictest0-21swapper/323:13:163
288806399200,19cyclictest0-21swapper/322:29:213
288806399200,19cyclictest0-21swapper/321:09:253
288806399200,19cyclictest0-21swapper/321:02:283
288806399200,19cyclictest0-21swapper/320:21:253
288806399200,19cyclictest0-21swapper/320:21:253
288806399200,19cyclictest0-21swapper/319:46:313
288806399200,19cyclictest0-21swapper/300:03:173
288806199208,4cyclictest34-21ksoftirqd/221:00:192
288806199205,5cyclictest34-21ksoftirqd/200:07:232
288806199204,2cyclictest141rcu_preempt23:20:002
288806199202,2cyclictest141rcu_preempt19:25:002
2888061992010,4cyclictest34-21ksoftirqd/220:30:202
2888061992010,3cyclictest34-21ksoftirqd/222:50:222
288806199200,19cyclictest0-21swapper/221:05:292
288805799209,2cyclictest141rcu_preempt22:50:471
288805799206,4cyclictest26-21ksoftirqd/120:25:001
288805799206,4cyclictest26-21ksoftirqd/120:25:001
288805799203,3cyclictest26-21ksoftirqd/100:00:011
2888057992018,1cyclictest3056-21PK-Backend22:16:141
2888057992018,1cyclictest3056-21PK-Backend22:16:141
2888057992017,2cyclictest251rcuc/123:37:061
2888057992017,2cyclictest251rcuc/119:10:341
288805799200,6cyclictest3056-21PK-Backend22:27:321
288805799200,6cyclictest3056-21PK-Backend22:25:001
288805799200,6cyclictest3056-21PK-Backend21:21:081
288805799200,3cyclictest26-21ksoftirqd/120:15:001
288805799200,19cyclictest3056-21PK-Backend21:57:011
288805799200,19cyclictest3056-21PK-Backend19:28:441
288805799200,18cyclictest3056-21PK-Backend20:05:011
288805499207,2cyclictest141rcu_preempt19:32:020
288805499205,3cyclictest13-21ksoftirqd/023:25:190
288805499203,3cyclictest13-21ksoftirqd/020:51:010
288805499202,7cyclictest13-21ksoftirqd/023:20:110
288805499202,4cyclictest13-21ksoftirqd/020:00:120
288805499201,5cyclictest161rcuc/022:02:550
288805499200,19cyclictest0-21swapper/021:25:570
2888063991917,1cyclictest0-21swapper/323:37:153
2888063991917,1cyclictest0-21swapper/322:08:263
2888063991917,1cyclictest0-21swapper/319:13:393
2888063991915,3cyclictest0-21swapper/320:00:193
2888063991913,5cyclictest2924137-21basename20:50:143
2888063991913,5cyclictest2916548-21systemd-run20:30:003
2888063991912,5cyclictest1-21systemd23:10:003
2888063991911,2cyclictest42-21ksoftirqd/300:35:013
288806399190,7cyclictest2917861-21/usr/sbin/munin20:30:213
288806399190,1cyclictest0-21swapper/322:33:133
288806399190,1cyclictest0-21swapper/320:42:283
288806399190,1cyclictest0-21swapper/320:11:253
288806399190,18cyclictest0-21swapper/321:50:213
288805799192,6cyclictest3056-21PK-Backend23:11:371
2888057991917,1cyclictest3056-21PK-Backend22:35:281
2888057991915,3cyclictest251rcuc/120:55:031
288805799190,6cyclictest3056-21PK-Backend21:10:331
288805799190,18cyclictest3056-21PK-Backend20:51:451
288806399189,3cyclictest141rcu_preempt21:20:023
288806399187,9cyclictest42-21ksoftirqd/322:20:173
288806399185,11cyclictest42-21ksoftirqd/322:45:163
288806399183,13cyclictest0-21swapper/321:40:173
2888063991816,1cyclictest3056-21PK-Backend23:41:073
288806399181,15cyclictest2997270-21df_inode00:15:123
2888063991810,5cyclictest0-21swapper/319:25:123
288805799184,2cyclictest141rcu_preempt19:39:441
2888057991816,1cyclictest3056-21PK-Backend21:05:101
288805799180,6cyclictest3056-21PK-Backend23:30:011
288805799170,6cyclictest3056-21PK-Backend00:01:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional