You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 20:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Jun 28, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1541429569,8sleep00-21swapper/007:06:590
1535929380,9sleep20-21swapper/207:06:162
1541028871,13sleep10-21swapper/107:06:551
1550428673,9sleep30-21swapper/307:08:093
15740993231,1cyclictest0-21swapper/209:16:252
1574099270,26cyclictest0-21swapper/210:15:202
1573899233,18cyclictest30940-21fschecks_time07:40:171
1573899230,1cyclictest0-21swapper/112:30:581
230662220,1sleep30-21swapper/309:18:313
1574399220,3cyclictest0-21swapper/308:20:133
1574099224,11cyclictest0-21swapper/210:00:132
1574099220,22cyclictest0-21swapper/209:46:172
1573899220,0cyclictest0-21swapper/109:52:541
1574399211,19cyclictest14719-21sh11:09:513
1574399210,1cyclictest0-21swapper/311:45:093
1574099210,20cyclictest0-21swapper/210:35:562
1573899216,13cyclictest0-21swapper/112:15:121
15738992020,0cyclictest0-21swapper/109:46:461
10802200,0sleep30-21swapper/311:59:203
15743991816,1cyclictest4055-21kworker/3:212:13:033
1574399180,17cyclictest0-21swapper/309:28:043
1574099181,16cyclictest30362-21pmu-power12:35:222
1573899181,3cyclictest13080-21/usr/sbin/munin12:25:191
1573899180,1cyclictest0-21swapper/110:57:011
15743991717,0cyclictest0-21swapper/310:52:513
15743991717,0cyclictest0-21swapper/310:52:513
1574399170,2cyclictest0-21swapper/310:05:163
1574399160,16cyclictest0-21swapper/312:33:303
1574099162,12cyclictest0-21swapper/211:23:372
1574099161,13cyclictest4614-21grep12:20:142
15738991614,1cyclictest0-21swapper/111:05:171
1573899160,1cyclictest0-21swapper/112:01:271
1573899160,1cyclictest0-21swapper/110:09:351
1573899160,15cyclictest8266-21diskmemload10:40:431
82322150,1sleep20-21swapper/210:26:312
1574099154,11cyclictest3651-21kworker/u8:010:45:382
15740991513,1cyclictest0-21swapper/210:30:592
15740991510,4cyclictest30644-21wc10:20:252
1573899151,8cyclictest225-21systemd-journal08:30:001
15738991514,1cyclictest0-21swapper/110:23:471
1573899150,1cyclictest0-21swapper/111:25:211
1573899150,14cyclictest0-21swapper/111:04:521
1573899150,14cyclictest0-21swapper/110:45:421
1573499153,6cyclictest22866-21munin-run07:25:000
1573499153,6cyclictest11678-21awk12:25:000
1574399144,6cyclictest20548-21ssh10:15:133
15743991413,1cyclictest4055-21kworker/3:212:20:133
15743991413,1cyclictest0-21swapper/311:16:023
15743991413,1cyclictest0-21swapper/309:49:053
15743991412,1cyclictest0-21swapper/309:54:403
15743991410,3cyclictest8351-21sed08:00:163
15743991410,3cyclictest6827-21grep07:55:263
15740991410,3cyclictest29332-21cut08:45:152
15740991410,3cyclictest26582-21tr11:35:212
15740991410,3cyclictest1094-21cron08:55:002
1574099140,14cyclictest0-21swapper/212:17:232
15738991410,3cyclictest19408-21ls07:15:201
1573899140,13cyclictest25834-21wc08:35:311
1573499143,6cyclictest31164-21munin-run08:50:000
1573499143,6cyclictest11619-21munin-run10:10:010
1573499140,1cyclictest0-21swapper/010:10:130
1574399139,3cyclictest932-21apt09:25:003
1574399139,3cyclictest3653-21cat09:00:003
1574399139,3cyclictest30794-21grep10:20:263
1574399139,3cyclictest2851-21wc12:00:153
1574399139,3cyclictest20920-21grep09:55:233
1574399139,3cyclictest19290-21ls11:50:183
15743991312,1cyclictest55650irq/127-enp2s0-09:44:543
15743991311,2cyclictest39-21ksoftirqd/309:39:003
15743991311,1cyclictest0-21swapper/309:11:513
1574399131,0cyclictest0-21swapper/310:28:053
15743991310,2cyclictest6426-21tr09:05:143
15743991310,2cyclictest4006-21grep09:00:163
15743991310,2cyclictest24372-21cat10:55:223
15743991310,2cyclictest18974-21sort07:15:163
15743991310,2cyclictest17667-21cron09:35:003
15743991310,2cyclictest16900-21grep11:10:223
15743991310,2cyclictest1166-21ls11:20:203
1574099139,3cyclictest23392-21wc08:30:272
1574099139,3cyclictest17276-21wc11:10:262
1574099139,2cyclictest12457-21tr12:25:162
1574099132,1cyclictest141rcu_preempt11:27:392
15740991312,1cyclictest0-21swapper/209:41:482
1574099131,0cyclictest141rcu_preempt09:50:302
15740991310,2cyclictest4878-21grep09:00:212
15740991310,2cyclictest2801-21wc08:55:272
15740991310,2cyclictest17625-21cron11:50:002
15740991310,2cyclictest12419-21tr10:10:142
1574099130,12cyclictest27175-21http12:15:002
1574099130,12cyclictest0-21swapper/212:05:212
1573899139,3cyclictest29737-21sort12:35:171
1573899139,2cyclictest31711-21tr08:50:121
1573899138,4cyclictest25720-21ssh09:58:461
1573899138,3cyclictest433-21tr07:45:141
1573899137,6cyclictest18810-21kworker/u8:111:34:571
1573899136,5cyclictest18141-21ssh11:50:121
1573899132,2cyclictest225-21systemd-journal09:20:011
15738991310,2cyclictest9802-21cut09:30:111
15738991310,2cyclictest12974-21cut10:10:161
1573899130,12cyclictest16118-21grep11:10:161
1573499137,5cyclictest15160-21ssh09:13:510
1573499133,6cyclictest1-21systemd10:37:250
1573499132,9cyclictest22719-21ssh10:55:110
15734991310,2cyclictest30051-21sh12:35:190
15734991310,2cyclictest24693-21cat07:25:260
15734991310,2cyclictest16367-21grep11:10:180
15734991310,2cyclictest14959-21grep08:15:140
15734991310,2cyclictest1237-21wc11:20:210
1573499130,12cyclictest22310-21cat10:55:000
1573499130,12cyclictest22310-21cat10:55:000
1573499130,0cyclictest0-21swapper/011:15:130
1574399129,2cyclictest28782-21sh08:45:013
1574399129,2cyclictest28593-21grep07:35:163
1574399129,2cyclictest27181-21head11:35:263
1574399129,2cyclictest23531-21grep08:30:273
1574399129,2cyclictest1505-21tr11:40:133
1574399129,2cyclictest12504-21grep12:25:153
1574399129,2cyclictest11654-21sed08:05:263
1574399128,3cyclictest17024-21grep11:30:153
1574399128,2cyclictest11127-21grep12:05:143
1574399126,2cyclictest452-21systemd-logind11:00:223
1574399125,6cyclictest8215-21apt-get11:25:133
1574399124,2cyclictest225-21systemd-journal07:35:003
1574399123,8cyclictest16991-21memory07:10:203
1574099129,2cyclictest7393-21grep11:05:142
1574099129,2cyclictest4862-21grep07:55:012
1574099129,2cyclictest31845-21uniq11:00:172
1574099129,2cyclictest30994-21grep07:40:172
1574099129,2cyclictest28266-21wc11:55:252
1574099129,2cyclictest25250-21ssh09:58:232
1574099129,2cyclictest2097-21sed11:40:162
1574099129,2cyclictest19299-21wc08:25:012
1574099129,2cyclictest16466-21grep08:15:262
1574099128,3cyclictest13042-21grep08:10:172
1574099124,4cyclictest225-21systemd-journal12:05:002
15740991210,1cyclictest26081-21sort11:55:002
1574099120,12cyclictest0-21swapper/209:26:152
1573899129,2cyclictest7179-21wc09:05:201
1573899129,2cyclictest29991-21dump-pmu-power07:40:001
1573899129,2cyclictest2608-21ls11:40:201
1573899129,2cyclictest25949-21cut11:35:151
1573899129,2cyclictest22143-21grep08:30:151
1573899129,2cyclictest13548-21grep08:10:231
1573899128,3cyclictest15763-21sed10:50:211
1573899128,3cyclictest15763-21sed10:50:201
1573899128,1cyclictest262882sleep110:37:311
1573899127,3cyclictest8266-21diskmemload10:01:581
1573899123,2cyclictest7019-21ssh11:24:291
15738991212,0cyclictest0-21swapper/110:26:151
1573899121,11cyclictest0-21swapper/111:19:281
1573899120,11cyclictest0-21swapper/109:23:471
1573899120,0cyclictest0-21swapper/109:44:071
1573499129,2cyclictest9348-21cat11:45:000
1573499129,2cyclictest7858-21grep11:05:170
1573499129,2cyclictest7844-21grep08:00:150
1573499129,2cyclictest5886-21head07:55:160
1573499129,2cyclictest3506-21perf09:00:010
1573499129,2cyclictest32647-21chrt12:17:370
1573499129,2cyclictest31833-21cat07:40:260
1573499129,2cyclictest29695-21head10:20:160
1573499129,2cyclictest22421-21grep10:15:240
1573499129,2cyclictest19479-21tr07:15:210
1573499129,2cyclictest1354-21tr07:45:190
1573499129,2cyclictest10623-21fschecks_time08:05:160
1573499128,3cyclictest6773-21grep09:05:160
1573499128,3cyclictest6039-21cat10:25:210
1573499128,3cyclictest1373-21taskset09:25:100
1573499128,2cyclictest6889-21ssh10:45:170
238372110,1sleep20-21swapper/211:15:122
1574399119,1cyclictest912-21grep07:45:173
1574399118,2cyclictest8285-21ssh10:45:413
1574399118,2cyclictest28976-21ssh12:35:133
1574399118,2cyclictest28625-21expr12:15:163
1574399118,2cyclictest21892-21ssh10:35:153
1574399118,2cyclictest21833-21grep07:20:193
1574399118,2cyclictest14653-21sh10:10:513
1574399118,2cyclictest14540-21ntp_states10:30:213
1574399117,4cyclictest26740-21munin-node07:29:473
1574399117,3cyclictest31065-21cat10:40:193
1574399110,10cyclictest31570-21rm10:01:473
1574099119,1cyclictest26980-21wc07:30:262
1574099119,1cyclictest25549-21cut09:20:122
1574099118,3cyclictest55650irq/127-enp2s0-12:30:452
1574099118,2cyclictest5483-21ssh10:05:252
1574099118,2cyclictest28303-21grep08:40:272
1574099118,2cyclictest25984-21ssh09:39:592
1574099118,2cyclictest24816-21grep07:25:272
1574099118,2cyclictest24303-21cut08:35:182
1574099118,2cyclictest18382-21grep07:15:142
1574099118,2cyclictest17508-21ssh11:30:172
1574099118,2cyclictest15146-21grep10:50:162
1574099118,2cyclictest15146-21grep10:50:162
1574099118,2cyclictest12572-21ssh09:31:032
1574099118,2cyclictest10753-21head09:10:272
1574099117,3cyclictest7470-21fgrep09:05:222
1574099113,4cyclictest225-21systemd-journal07:15:002
1574099111,9cyclictest0-21swapper/210:55:112
1574099111,1cyclictest0-21swapper/210:40:132
1573899118,2cyclictest9697-21basename08:04:591
1573899118,2cyclictest5795-21grep07:55:161
1573899118,2cyclictest4528-21cut09:00:181
1573899118,2cyclictest3616-21wc09:00:001
1573899118,2cyclictest29730-21sort08:45:171
1573899118,2cyclictest27537-21sh11:55:171
1573899118,2cyclictest2706-21grep09:25:211
1573899118,2cyclictest26715-21head07:30:221
1573899118,2cyclictest24844-21ssh10:17:141
1573899118,2cyclictest23719-21ssh12:12:211
1573899118,2cyclictest17919-21ssh09:35:011
1573899118,2cyclictest13385-21cut10:30:151
1573899118,2cyclictest10668-21grep09:10:261
1573899117,3cyclictest10563-21tr11:45:171
15738991110,1cyclictest3782-21apt-get12:20:111
1573899110,8cyclictest225-21systemd-journal07:15:001
1573499119,1cyclictest26731-21grep09:20:180
1573499119,1cyclictest20182-21cut09:55:170
1573499119,1cyclictest17652-21grep07:10:270
1573499118,2cyclictest3369-21head07:50:160
1573499118,2cyclictest3345-21grep12:00:170
1573499118,2cyclictest3342-21ssh10:43:220
1573499118,2cyclictest29105-21ssh09:40:410
1573499118,2cyclictest28710-21ssh11:55:340
1573499118,2cyclictest27787-21cat10:00:110
1573499118,2cyclictest26676-21ssh11:35:210
1573499118,2cyclictest26137-21tune2fs07:30:170
1573499118,2cyclictest24917-21ssh09:39:090
1573499118,2cyclictest24301-21ssh12:31:460
1573499118,2cyclictest23004-21cat08:30:210
1573499118,2cyclictest20747-21ssh11:50:540
1573499118,2cyclictest18216-21cat08:20:190
1573499118,2cyclictest14695-21chrt09:51:420
1573499118,2cyclictest13103-21head08:10:170
1573499118,2cyclictest11177-21ssh11:26:010
1573499118,2cyclictest10319-21df11:45:160
1573499117,3cyclictest3921-21cat09:45:190
1573499117,3cyclictest19411-21ssh11:30:510
1573499117,3cyclictest14670-21ssh12:06:520
1573499117,2cyclictest19406-21cut12:10:140
1573499116,3cyclictest27460-21tail08:40:180
1574399108,1cyclictest9902sleep308:53:433
1574399108,1cyclictest4389-21taskset07:51:093
1574399108,1cyclictest31043-21taskset08:48:393
1574399108,1cyclictest30476-21cpuspeed_turbos07:40:143
1574399108,1cyclictest24529-21expr08:35:193
1574399108,1cyclictest19671-21wget08:25:143
1574399108,1cyclictest141952sleep308:12:083
1574399107,2cyclictest15074-21cat08:15:143
1574099108,1cyclictest7092-21taskset07:57:392
1574099107,2cyclictest10627-21sort08:05:162
1573899108,1cyclictest4901-21mktemp07:55:011
1573899107,2cyclictest23170-21cut07:25:161
1573899107,2cyclictest12053-21grep12:05:201
1573899105,3cyclictest14664-21/usr/sbin/munin08:15:121
1573899102,4cyclictest225-21systemd-journal07:20:001
1573899101,5cyclictest225-21systemd-journal08:25:001
1573499108,1cyclictest31412-21basename08:50:090
1573499108,1cyclictest13227-21chrt10:30:130
1573499108,1cyclictest12376-21grep12:25:150
1573499107,2cyclictest28200-21expr07:35:140
1573499107,2cyclictest25293-21cat08:35:240
1573499107,2cyclictest20309-21expr08:25:170
1573499107,2cyclictest20035-21sh09:16:080
1573499107,2cyclictest11285-21grep09:30:200
1573499100,9cyclictest5805-21seq09:04:100
1573499100,9cyclictest30881-21ssh11:00:020
157409997,1cyclictest847-21tune2fs07:45:162
157409997,1cyclictest29939-21chrt07:39:422
157409997,1cyclictest22799-21taskset07:24:282
157409997,1cyclictest20993-21sed08:25:252
157409996,2cyclictest8361-21head08:00:162
157389996,2cyclictest27281-21sort08:40:171
157389992,6cyclictest10179-21tr08:05:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional