You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 08:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Jul 20, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1867329481,9sleep20-21swapper/219:06:152
1893528673,9sleep00-21swapper/019:09:390
1885828464,16sleep10-21swapper/119:08:391
1866228168,9sleep30-21swapper/319:06:083
139972580,0sleep00-21swapper/022:08:290
1904099261,24cyclictest0-21swapper/122:25:001
19045992424,0cyclictest0-21swapper/200:35:342
1904599230,23cyclictest0-21swapper/223:11:162
1904599230,1cyclictest0-21swapper/200:00:152
1904599230,10cyclictest0-21swapper/223:18:252
1904099231,21cyclictest31257-21fschecks_time23:55:161
19050992120,1cyclictest0-21swapper/323:06:143
19040992117,2cyclictest0-21swapper/121:28:481
1904099210,21cyclictest0-21swapper/122:29:091
19036992119,2cyclictest13-21ksoftirqd/000:19:420
19040992019,1cyclictest0-21swapper/123:43:091
19040992018,1cyclictest0-21swapper/123:45:071
19050991915,2cyclictest12102-21diskmemload22:19:423
1904599193,11cyclictest1142-21runrttasks23:44:242
1904599192,5cyclictest1142-21runrttasks00:21:262
1904599190,18cyclictest0-21swapper/222:46:262
19036991919,0cyclictest0-21swapper/021:26:020
19040991813,4cyclictest27721-21ssh21:57:001
1904099180,1cyclictest0-21swapper/121:49:491
1904099180,1cyclictest0-21swapper/100:20:071
1905099170,16cyclictest0-21swapper/300:11:023
19045991715,1cyclictest0-21swapper/221:50:142
1904599170,17cyclictest0-21swapper/223:31:282
19036991716,1cyclictest0-21swapper/000:28:120
1905099161,10cyclictest0-21swapper/300:25:003
1904099162,11cyclictest1142-21runrttasks21:52:021
19040991614,1cyclictest0-21swapper/122:55:201
19040991612,2cyclictest0-21swapper/122:00:581
1904099160,16cyclictest0-21swapper/122:36:261
1903699160,4cyclictest30463-21/usr/sbin/munin23:55:220
19050991514,1cyclictest0-21swapper/323:38:123
19050991512,3cyclictest0-21swapper/323:34:263
1905099150,1cyclictest0-21swapper/323:45:513
1905099150,13cyclictest0-21swapper/300:28:353
1904599150,15cyclictest0-21swapper/222:36:412
1904599150,14cyclictest0-21swapper/221:19:162
1904599150,14cyclictest0-21swapper/221:19:162
1904099159,5cyclictest28293-21/usr/sbin/munin23:15:201
19040991514,1cyclictest0-21swapper/123:50:061
1904099150,1cyclictest0-21swapper/122:16:091
19036991512,1cyclictest0-21swapper/021:10:540
1903699150,1cyclictest0-21swapper/021:55:150
1905099145,6cyclictest8284-21latency_hist19:55:013
19050991412,2cyclictest39-21ksoftirqd/323:22:003
19050991412,2cyclictest39-21ksoftirqd/323:19:573
19050991412,2cyclictest39-21ksoftirqd/321:57:183
19050991411,2cyclictest24606-21df19:20:173
19050991411,2cyclictest24606-21df19:20:163
19050991410,3cyclictest15488-21grep21:30:223
1905099140,1cyclictest0-21swapper/322:07:583
19045991411,3cyclictest16804-1kworker/u9:123:50:132
19045991410,3cyclictest32-21ksoftirqd/220:49:592
1904599140,1cyclictest0-21swapper/223:49:142
1904599140,13cyclictest0-21swapper/222:06:292
1904599140,13cyclictest0-21swapper/221:33:032
1904099147,5cyclictest22758-21cat23:30:261
19040991413,1cyclictest0-21swapper/100:34:061
19040991411,2cyclictest29069-21tr19:30:151
19040991410,4cyclictest3370-21apache223:28:591
19040991410,3cyclictest3366-21apache200:07:131
19040991410,3cyclictest25450-21sh22:35:001
19040991410,2cyclictest10115-21grep21:05:151
1903699146,6cyclictest0-21swapper/022:00:200
1903699143,7cyclictest27297-21sshd20:31:300
1903699140,14cyclictest0-21swapper/022:59:330
1905099137,4cyclictest3021-21latency22:40:193
19050991311,2cyclictest3366-21apache223:53:343
19050991310,2cyclictest23298-21sed19:15:283
19050991310,2cyclictest18690-21cat22:30:193
19050991310,2cyclictest12057-21wc22:45:273
1904599138,3cyclictest3820-21sh23:39:102
1904599134,5cyclictest225-21systemd-journal21:50:002
1904599133,9cyclictest32741-21/usr/sbin/munin22:00:142
1904599133,9cyclictest0-21swapper/221:20:282
1904599132,8cyclictest25305-21/usr/sbin/munin00:30:172
1904599132,8cyclictest21394-21munin-run19:14:592
19045991310,2cyclictest12078-21cron21:10:002
1904599130,13cyclictest0-21swapper/223:26:172
1904599130,13cyclictest0-21swapper/222:21:322
1904599130,13cyclictest0-21swapper/221:59:552
1904099139,3cyclictest883-21grep20:45:151
1904099139,3cyclictest31332-21grep20:40:191
1904099138,4cyclictest28531-21cron19:30:001
1904099136,6cyclictest0-21swapper/123:38:011
1904099132,1cyclictest12595-21apache_accesses21:10:131
19040991310,2cyclictest9782-21wc22:05:271
19040991310,2cyclictest8433-21grep00:00:291
19040991310,2cyclictest24344-21tr19:20:161
19040991310,2cyclictest24344-21tr19:20:161
19040991310,2cyclictest22149-21df19:15:171
19040991310,2cyclictest11169-21sed22:45:171
19040991310,2cyclictest11074-21cut20:00:131
1904099130,12cyclictest7346-21sleep00:20:001
19036991310,2cyclictest30834-21wc23:35:210
19036991310,2cyclictest2725-21tail22:20:260
19036991310,2cyclictest26558-21grep00:30:270
19036991310,2cyclictest18101-21cut20:15:140
1903699130,13cyclictest0-21swapper/021:50:140
293572120,1chrt39-21ksoftirqd/322:36:333
1905099129,2cyclictest3976-21ssh00:17:243
1905099129,2cyclictest27210-21head19:25:183
1905099129,2cyclictest26707-21grep20:30:233
1905099129,2cyclictest20880-21tail23:10:173
1905099129,2cyclictest14622-21sh21:10:273
1905099129,2cyclictest13938-21grep20:05:223
1905099128,3cyclictest19769-21ssh22:11:263
1905099128,3cyclictest0-21swapper/321:44:153
1904599129,2cyclictest8647-21wc21:00:252
1904599129,2cyclictest5069-21ssh23:20:172
1904599129,2cyclictest4041-21tr19:45:122
1904599129,2cyclictest32275-21sh00:15:152
1904599129,2cyclictest24662-21sed21:35:292
1904599129,2cyclictest18145-21wc22:10:252
1904599129,2cyclictest1427-21tr19:40:162
1904599129,2cyclictest10672-21perf20:00:002
1904599128,3cyclictest31879-21ssh21:40:162
1904599128,2cyclictest0-21swapper/200:09:532
1904599123,5cyclictest12102-21diskmemload22:42:482
19045991210,1cyclictest6002-21apt00:00:002
19045991210,1cyclictest25684-21grep19:20:282
19045991210,1cyclictest25684-21grep19:20:282
19045991210,1cyclictest17829-21taskset00:25:242
1904599120,1cyclictest0-21swapper/223:01:552
1904599120,11cyclictest29535-21grep20:35:282
1904099129,2cyclictest940-21expr00:35:131
1904099129,2cyclictest8615-21grep19:55:161
1904099129,2cyclictest5716-21tail20:55:181
1904099129,2cyclictest31384-21cron00:14:591
1904099129,2cyclictest28129-21tr20:35:161
1904099129,2cyclictest2806-21tail19:40:271
1904099129,2cyclictest254692sleep120:30:131
1904099129,2cyclictest23899-21tr21:35:211
1904099129,2cyclictest19523-21cut19:10:141
1904099128,3cyclictest5098-21grep23:20:171
1904099128,2cyclictest6305-21wget19:50:141
1904099127,3cyclictest27857-21munin-run23:15:011
1904099126,5cyclictest12369-21basename23:05:191
1904099126,2cyclictest2174-21tr22:40:141
1904099120,12cyclictest0-21swapper/121:20:161
1903699129,2cyclictest8489-21munin-plugin-st00:40:000
1903699129,2cyclictest32336-21cron20:45:000
1903699129,2cyclictest25220-21sleep20:30:000
1903699129,2cyclictest24614-21tail21:35:280
1903699129,2cyclictest24333-21grep00:10:170
1903699129,2cyclictest19289-21cron23:10:000
1903699129,2cyclictest15663-21ssh00:24:590
1903699129,2cyclictest14852-21cron21:50:010
1903699129,2cyclictest14621-21grep23:25:280
19036991210,1cyclictest10624-21grep21:05:170
1903699120,11cyclictest7985-21head00:00:240
1905099119,1cyclictest7430-21tr21:00:173
1905099118,2cyclictest9152-21ssh22:25:113
1905099118,2cyclictest5391-21ssh21:25:023
1905099118,2cyclictest4281-21ssh00:36:453
1905099118,2cyclictest32136-21wc19:35:213
1905099118,2cyclictest30846-21ssh22:56:493
1905099118,2cyclictest29739-21ssh21:20:173
1905099118,2cyclictest2416-21ssh22:20:243
1905099118,2cyclictest22385-21sed20:20:253
1905099118,2cyclictest21049-21ssh21:15:143
1905099118,2cyclictest21049-21ssh21:15:143
1905099118,2cyclictest20352sleep320:46:443
1905099118,2cyclictest199062sleep320:16:363
1905099118,2cyclictest1699-21ssh23:56:423
1905099118,2cyclictest14808-21ssh00:05:103
1905099118,2cyclictest12381-21ssh23:44:183
1905099118,2cyclictest11281-21expr20:00:143
1905099117,3cyclictest29791-21wc19:30:213
1905099117,3cyclictest23001-21ssh21:35:173
1905099113,6cyclictest375-21sort22:00:163
1905099113,4cyclictest225-21systemd-journal19:49:563
1905099112,2cyclictest0-21swapper/320:35:143
19050991110,1cyclictest0-21swapper/321:46:073
1904599119,1cyclictest24205-21uname20:25:202
1904599119,1cyclictest22191-21wc20:20:252
1904599119,1cyclictest19032-21sh22:30:232
1904599118,2cyclictest5417-21expr20:55:162
1904599118,2cyclictest32451-21sh22:19:492
1904599118,2cyclictest3054-21grep20:50:162
1904599118,2cyclictest26826-21sh22:55:032
1904599118,2cyclictest26188-21expr20:30:182
1904599118,2cyclictest23420-21ssh00:10:082
1904599118,2cyclictest22983-21wc19:15:242
1904599118,2cyclictest19495-21tail22:50:172
1904599118,2cyclictest18257-21sh21:13:162
1904599118,2cyclictest16781-21tail20:10:202
1904599118,2cyclictest13562-21sh22:27:162
1904599114,6cyclictest11542-21aten2_r7power_e23:05:152
1904599113,2cyclictest141rcu_preempt19:30:002
1904099118,2cyclictest9092-21grep21:00:281
1904099118,2cyclictest4418-21cut19:45:141
1904099118,2cyclictest4405-21ssh23:00:221
1904099118,2cyclictest3300-21sh21:42:341
1904099118,2cyclictest2665-21wget20:50:141
1904099118,2cyclictest19724-21tail20:15:271
1904099118,2cyclictest19097-21ssh00:26:081
1904099111,5cyclictest0-21swapper/119:35:151
1904099111,2cyclictest225-21systemd-journal21:20:001
1904099111,2cyclictest225-21systemd-journal21:20:001
1904099111,2cyclictest225-21systemd-journal20:10:001
1904099110,10cyclictest20836-21ssh22:50:331
1903699119,1cyclictest19141-21expr22:30:230
1903699118,2cyclictest29210-21sed23:15:170
1903699118,2cyclictest2673-21grep22:40:160
1903699118,2cyclictest21660-21ssh00:08:570
1903699118,2cyclictest19848-21cat22:50:200
1903699118,2cyclictest16153-21grep21:30:290
1903699118,2cyclictest13986-21tune2fs20:05:220
1903699118,2cyclictest12161-21grep20:00:200
1903699118,2cyclictest10645-21cron20:00:010
1903699118,2cyclictest10596-21cut22:45:150
1903699115,2cyclictest0-21swapper/022:14:430
1903699114,7cyclictest0-21swapper/021:17:530
1903699114,7cyclictest0-21swapper/021:17:530
1903699113,6cyclictest0-21swapper/021:20:010
1903699113,3cyclictest25986-21ssh22:35:120
1903699110,1cyclictest0-21swapper/021:44:160
1903699110,10cyclictest24737-21uniq19:20:180
1903699110,10cyclictest24737-21uniq19:20:170
1905099108,2cyclictest16705-21ssh21:50:213
1905099108,1cyclictest6465-21ssh00:00:113
1905099108,1cyclictest25794-21load00:30:203
1905099108,1cyclictest23483-21expr20:25:153
1905099108,1cyclictest185322sleep322:50:073
1905099108,1cyclictest10519-21df_abs21:05:163
1905099107,2cyclictest6611-21grep20:55:283
1905099107,2cyclictest4492-21grep23:00:233
1905099107,2cyclictest20374-21cat19:10:203
1905099107,2cyclictest13967-21uname23:25:213
1905099100,9cyclictest30225-21bc20:40:113
1905099100,1cyclictest0-21swapper/319:40:133
1904599108,2cyclictest31224-21wc20:40:192
1904599108,1cyclictest7593-21grep21:25:262
1904599108,1cyclictest7187-21load19:50:192
1904599108,1cyclictest287702sleep219:30:112
1904599108,1cyclictest180232sleep220:15:032
1904599107,2cyclictest32148-21sed19:35:212
1904599107,2cyclictest13657-21cat20:05:202
1904099108,1cyclictest23101-21apt-get20:25:131
1904099108,1cyclictest20313-21sh22:11:521
1904099108,1cyclictest14140-21ssh21:30:111
1904099107,2cyclictest21401-21grep20:20:171
1904099106,3cyclictest16459-21sed20:10:161
1903699108,2cyclictest783-21taskset19:39:180
1903699108,2cyclictest22246-21timerwakeupswit20:20:260
1903699108,2cyclictest11798-21ssh22:25:530
1903699108,1cyclictest7295-21date21:00:000
1903699108,1cyclictest6230-21basename19:50:140
1903699108,1cyclictest4093-21sh23:20:010
1903699108,1cyclictest3635-21load20:50:190
1903699108,1cyclictest25839-21ssh22:15:190
1903699108,1cyclictest25605-21ssh23:13:140
1903699108,1cyclictest23501-21awk23:50:210
1903699108,1cyclictest20034-21grep19:10:170
1903699108,1cyclictest14070-21cut23:45:150
1903699107,2cyclictest9516-21awk21:04:590
1903699107,2cyclictest7322-21grep23:40:260
1903699107,2cyclictest3659-21cat19:45:000
1903699107,2cyclictest29028-21cut19:30:160
1903699107,2cyclictest28588-21grep20:35:170
1903699107,2cyclictest22800-21cat23:30:270
1903699107,2cyclictest22035-21expr19:15:160
1903699101,5cyclictest225-21systemd-journal19:45:010
1903699100,9cyclictest2757-21ssh23:00:020
190509998,1cyclictest4201-21cat20:50:273
190509997,1cyclictest8797-21grep19:55:163
190509997,1cyclictest156442sleep320:10:033
190459997,1cyclictest12474-21wc20:00:252
190369998,1cyclictest283972sleep019:28:210
190369997,1cyclictest32582-21basename20:45:090
190369997,1cyclictest16314-21cut20:10:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional