You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 17:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri Apr 24, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1321928760,9sleep30-21swapper/307:07:523
1314128674,8sleep20-21swapper/207:06:512
1315028467,13sleep10-21swapper/107:06:581
1311128464,9sleep00-21swapper/007:06:290
109532550,0sleep10-21swapper/111:27:521
1348099260,26cyclictest0-21swapper/011:37:430
1348699250,1cyclictest0-21swapper/212:16:012
13482992423,1cyclictest0-21swapper/109:37:321
13488992221,1cyclictest0-21swapper/309:13:033
1348699220,22cyclictest0-21swapper/211:13:422
1348299220,21cyclictest0-21swapper/110:04:261
13480992220,1cyclictest0-21swapper/012:07:240
1348699200,1cyclictest0-21swapper/209:40:352
13482992020,0cyclictest0-21swapper/110:26:071
13482992019,1cyclictest0-21swapper/108:30:241
13482992017,2cyclictest0-21swapper/111:11:561
1348299201,18cyclictest11252-21ssh12:25:421
1348899190,1cyclictest0-21swapper/310:26:433
1348699190,19cyclictest0-21swapper/210:37:122
1348699190,19cyclictest0-21swapper/209:15:322
13480991917,2cyclictest0-21swapper/009:56:300
1348099190,5cyclictest141rcu_preempt12:30:030
1348699180,1cyclictest0-21swapper/210:24:072
1348699180,18cyclictest0-21swapper/209:13:522
1348299181,16cyclictest0-21swapper/110:31:251
1348099185,7cyclictest13693-21munin-run11:30:000
1348099184,7cyclictest11701-21munin-run10:50:000
1348099182,2cyclictest141rcu_preempt09:20:110
13488991714,1cyclictest0-21swapper/311:06:223
1348699171,14cyclictest0-21swapper/210:04:382
1348699170,17cyclictest0-21swapper/211:27:232
1348699170,15cyclictest0-21swapper/210:34:012
1348299170,1cyclictest0-21swapper/108:55:121
13480991715,1cyclictest0-21swapper/010:16:290
1348099170,15cyclictest0-21swapper/007:10:120
1348899163,7cyclictest684-21ls07:50:003
1348899160,4cyclictest0-21swapper/312:19:513
1348699161,1cyclictest141rcu_preempt12:02:222
1348699161,1cyclictest141rcu_preempt12:02:222
1348699160,2cyclictest11602-21ntp_states10:10:202
1348699160,16cyclictest0-21swapper/212:11:162
13482991614,1cyclictest0-21swapper/112:06:291
13482991613,3cyclictest0-21swapper/112:00:151
13482991613,3cyclictest0-21swapper/112:00:151
1348299160,13cyclictest0-21swapper/110:43:411
13480991611,4cyclictest13183-21apache_processe11:10:120
1348099160,0cyclictest0-21swapper/012:02:510
1348099160,0cyclictest0-21swapper/012:02:510
1348899156,6cyclictest5785-21apache208:20:133
1348899153,8cyclictest0-21swapper/311:40:003
13488991514,1cyclictest0-21swapper/310:20:263
1348899151,2cyclictest0-21swapper/310:35:143
1348899150,13cyclictest0-21swapper/311:42:503
13486991515,0cyclictest0-21swapper/212:23:532
13486991515,0cyclictest0-21swapper/210:59:162
13486991514,1cyclictest0-21swapper/211:37:432
13486991510,4cyclictest5785-21apache208:50:132
1348299153,9cyclictest23541-21taskset10:56:141
13482991513,1cyclictest0-21swapper/110:05:471
13482991512,2cyclictest25-21ksoftirqd/107:25:011
1348299150,1cyclictest0-21swapper/109:42:581
1348099151,8cyclictest0-21swapper/011:00:130
1348099150,15cyclictest0-21swapper/012:14:370
1348899149,2cyclictest19471-21ssh11:13:483
1348899140,12cyclictest9606-21grep12:25:153
13486991413,1cyclictest0-21swapper/210:50:222
1348699140,14cyclictest0-21swapper/212:28:522
1348699140,13cyclictest0-21swapper/209:58:522
13482991413,1cyclictest6689-21diskmemload12:32:191
13482991412,1cyclictest25-21ksoftirqd/112:11:351
13482991410,4cyclictest38150irq/131-snd_hda08:10:141
1348099147,6cyclictest5785-21apache210:40:170
1348099147,6cyclictest20026-21munin-run08:30:000
13480991411,2cyclictest2608-21ssh09:45:250
13480991410,2cyclictest25696-21grep09:40:160
1348099140,2cyclictest0-21swapper/011:43:420
1348099140,0cyclictest0-21swapper/010:39:420
1348899139,3cyclictest29339-21apt11:20:003
1348899139,3cyclictest26692-21sed08:40:263
13488991310,2cyclictest8518-21awk12:25:013
13488991310,2cyclictest6569-21sort11:25:163
13488991310,2cyclictest25773-21awk07:35:003
13488991310,2cyclictest14746-21grep08:15:263
13488991310,2cyclictest1039-21grep09:45:143
1348899130,5cyclictest0-21swapper/312:38:323
1348899130,4cyclictest0-21swapper/310:51:063
1348899130,12cyclictest6053-21grep08:00:143
1348899130,12cyclictest16559-21grep09:15:163
1348699138,4cyclictest32-21ksoftirqd/212:32:562
13486991312,1cyclictest0-21swapper/211:49:202
13486991310,2cyclictest32292-21head11:40:212
1348699130,13cyclictest0-21swapper/211:31:162
1348299132,0cyclictest141rcu_preempt11:18:401
13482991312,1cyclictest0-21swapper/109:46:311
1348299131,12cyclictest6689-21diskmemload10:45:161
1348299131,11cyclictest0-21swapper/112:17:431
13482991310,2cyclictest32681-21grep09:25:141
13482991310,2cyclictest2456-21grep12:20:261
13482991310,2cyclictest18205-21cut09:55:141
1348299130,12cyclictest30316-21grep07:40:261
1348299130,12cyclictest27491-21tr08:45:141
1348299130,12cyclictest27491-21tr08:45:141
1348099139,3cyclictest9465-21sort09:30:180
1348099139,3cyclictest24370-21head07:30:160
1348099139,3cyclictest12364-21grep08:10:260
1348099139,2cyclictest9980-21grep09:50:160
13480991310,2cyclictest9593-21munin-plugin-st10:10:000
13480991310,2cyclictest16410-21ls11:50:200
1348099130,8cyclictest0-21swapper/009:05:140
1348099130,2cyclictest0-21swapper/011:30:260
1348099130,1cyclictest0-21swapper/008:35:120
118302130,1sleep10-21swapper/109:31:401
245772120,1sleep26689-21diskmemload10:18:332
1348899129,2cyclictest5432-21swap10:45:243
1348899129,2cyclictest30928-21cat07:45:003
1348899129,2cyclictest28358-21dump-pmu-power11:00:003
1348899129,2cyclictest27003-21sed07:35:163
1348899129,2cyclictest22974-21grep07:25:253
1348899129,2cyclictest18439-21readlink07:20:003
1348899129,2cyclictest175422sleep311:50:423
1348899129,2cyclictest1740-21head07:50:153
1348899129,2cyclictest117172sleep308:10:193
1348899128,2cyclictest32694-21latency_hist09:45:003
1348899125,4cyclictest27952-21apt-get10:40:133
1348899122,4cyclictest225-21systemd-journal09:35:003
1348899120,11cyclictest0-21swapper/311:20:123
1348699129,2cyclictest4234-21grep10:25:202
1348699129,2cyclictest32486-21cat07:45:252
1348699129,2cyclictest31893-21idleruntime-cro09:25:002
1348699129,2cyclictest30772-21cat07:45:002
1348699129,2cyclictest30769-21sed11:00:252
1348699129,2cyclictest29662-21chrt12:37:052
1348699129,2cyclictest24613-21cat07:30:182
1348699129,2cyclictest23010-21wc11:15:212
1348699129,2cyclictest20027-21sleep08:30:002
1348699129,2cyclictest1912-21grep09:25:272
1348699129,2cyclictest17949-21ls09:35:212
1348699129,2cyclictest17120-21sed11:50:272
1348699125,4cyclictest1678-21/usr/sbin/munin09:45:172
13486991211,1cyclictest32-21ksoftirqd/210:47:262
13486991210,1cyclictest21419-21tr07:25:132
1348699120,11cyclictest32197-21grep08:55:122
1348699120,11cyclictest0-21swapper/208:30:132
1348299129,2cyclictest5036-21grep07:55:261
1348299129,2cyclictest31716-21head07:45:161
1348299129,2cyclictest29704-21grep11:00:171
1348299129,2cyclictest17525-21tr07:15:221
1348299129,2cyclictest12885-21grep10:50:161
1348299129,2cyclictest11136-21tail10:10:161
1348299128,3cyclictest31784-21wc11:40:171
1348299128,3cyclictest25-21ksoftirqd/109:05:141
1348299128,3cyclictest21355-21cat10:35:261
1348299127,4cyclictest27431-21ssh09:21:301
1348299125,6cyclictest25-21ksoftirqd/110:20:171
1348299124,6cyclictest0-21swapper/111:08:101
13482991211,1cyclictest0-21swapper/111:32:371
1348299120,11cyclictest4265-21perf09:05:001
1348299120,0cyclictest141rcu_preempt11:53:011
1348299120,0cyclictest0-21swapper/111:49:251
1348299120,0cyclictest0-21swapper/111:20:261
1348099129,2cyclictest31790-21sed07:45:160
1348099129,2cyclictest28348-21cron11:00:000
1348099129,2cyclictest25929-21grep10:00:130
1348099129,2cyclictest21511-21head08:30:200
1348099129,2cyclictest17655-21sleep08:25:000
1348099129,2cyclictest13368-21tr08:15:140
1348099129,2cyclictest12034-21tail10:30:150
1348099128,3cyclictest28704-21grep07:40:140
1348099128,3cyclictest25136-21wc11:55:250
1348099128,2cyclictest5926-21cut08:00:150
1348099128,2cyclictest27229-21tr10:20:140
1348099123,8cyclictest2783-21/usr/sbin/munin10:25:190
1348099120,2cyclictest0-21swapper/011:15:150
1348899118,2cyclictest9986-21head09:50:163
1348899118,2cyclictest6729-21ssh11:45:073
1348899118,2cyclictest473-21df09:25:153
1348899118,2cyclictest32040-21cut12:00:123
1348899118,2cyclictest32040-21cut12:00:123
1348899118,2cyclictest29057-21sed08:45:263
1348899118,2cyclictest29057-21sed08:45:263
1348899118,2cyclictest28965-21ssh11:00:133
1348899118,2cyclictest27459-21ssh10:00:243
1348899118,2cyclictest26665-21ssh11:56:223
1348899118,2cyclictest23985-21ssh09:20:083
1348899118,2cyclictest23379-21head08:35:163
1348899118,2cyclictest21265-21ssh10:15:563
1348899118,2cyclictest20871-21date07:25:013
1348899118,2cyclictest19905-21ssh09:55:473
1348899118,2cyclictest18466-21tr12:30:193
1348899118,2cyclictest18277-21grep12:10:273
1348899118,2cyclictest17763-21taskset09:35:193
1348899118,2cyclictest17681-21ssh11:31:363
1348899118,2cyclictest13941-21cut07:10:123
1348899118,2cyclictest13335-21ssh10:30:303
1348899118,2cyclictest10208-21ssh12:05:463
1348899117,3cyclictest2527-21grep10:05:163
1348899110,10cyclictest11371-21cat10:10:183
1348699119,1cyclictest25459-21grep11:55:262
1348699119,1cyclictest1984-21grep10:05:132
1348699118,2cyclictest8377-21apache_processe08:05:122
1348699118,2cyclictest5453-21uname09:05:202
1348699118,2cyclictest3982-21ssh11:24:002
1348699118,2cyclictest29554-21wc10:40:242
1348699118,2cyclictest2917-21grep09:00:162
1348699118,2cyclictest24066-21cat08:35:252
1348699118,2cyclictest20801-21sh07:25:002
1348699118,2cyclictest17533-21grep07:15:222
1348699118,2cyclictest11105-21sed09:50:252
1348699116,4cyclictest12850-21cron08:15:012
1348699114,3cyclictest225-21systemd-journal08:50:002
1348699114,3cyclictest225-21systemd-journal08:50:002
1348299118,2cyclictest7969-21head09:10:161
1348299118,2cyclictest26038-21cat08:40:191
1348299118,2cyclictest25083-21ssh11:35:521
1348299118,2cyclictest19621-21grep08:25:261
1348299118,2cyclictest18807-21ls10:15:121
1348299118,2cyclictest17193-21grep08:20:251
1348299118,2cyclictest15635-21chrt09:15:081
1348299118,2cyclictest13903-21grep08:15:161
1348299118,2cyclictest11108-21grep09:50:251
1348299110,10cyclictest1796-21cat07:50:161
1348299110,10cyclictest15248-21systemctl07:10:211
1348099119,1cyclictest27019-21tail07:35:170
1348099118,2cyclictest8619-21tr08:05:150
1348099118,2cyclictest8494-21cron12:25:000
1348099118,2cyclictest453-21cut08:55:150
1348099118,2cyclictest4074-21head07:55:160
1348099118,2cyclictest3682-21wc09:00:250
1348099118,2cyclictest31405-21sed08:50:230
1348099118,2cyclictest29684-21ssh12:17:490
1348099118,2cyclictest29065-21sed08:45:270
1348099118,2cyclictest29065-21sed08:45:260
1348099118,2cyclictest2061-21grep07:50:180
1348099118,2cyclictest17744-21wc07:15:260
1348099118,2cyclictest17636-21tail09:15:250
1348099118,2cyclictest1729-21wc09:25:260
1348099118,2cyclictest14105-21tail10:50:250
1348099118,2cyclictest13-21ksoftirqd/012:27:280
1348099118,2cyclictest11189-21ssh11:47:150
1348099117,3cyclictest26203-21ssh12:35:160
1348099115,2cyclictest16598-21apt-get09:35:130
1348099113,4cyclictest225-21systemd-journal08:45:000
1348899108,2cyclictest21578-21awk08:30:203
1348899108,1cyclictest6391-21chrt09:06:243
1348899108,1cyclictest3806-21expr07:55:143
1348899108,1cyclictest30256-21expr08:50:123
1348899108,1cyclictest1788-21taskset08:59:203
1348899107,2cyclictest19570-21sed08:25:263
1348699109,1cyclictest0-21swapper/212:05:132
1348699108,2cyclictest9334-21sh11:07:232
1348699108,1cyclictest8190-21sh09:30:082
1348699108,1cyclictest6017-21grep08:00:142
1348699108,1cyclictest5044-21grep07:55:262
1348699108,1cyclictest27475-21grep07:35:212
1348699108,1cyclictest17225-21grep08:20:252
1348699108,1cyclictest14636-21tail07:10:162
1348699107,2cyclictest25467-21expr08:40:152
1348699106,3cyclictest3063-21cron07:55:002
1348299108,1cyclictest30198-21expr08:50:121
1348299108,1cyclictest25221-21wc07:30:251
1348299108,1cyclictest23468-21seq11:55:111
1348299108,1cyclictest23233-21chrt07:28:091
1348299108,1cyclictest22911-21cut08:35:141
1348299107,2cyclictest9906-21cat08:05:261
1348299107,2cyclictest25670-21grep12:35:141
1348099108,1cyclictest11345-21cut10:10:180
1348099107,2cyclictest6301-21wc11:05:220
1348099107,2cyclictest22012-21expr07:25:160
1348099107,2cyclictest19035-21expr07:20:150
1348099101,2cyclictest0-21swapper/011:20:000
1348099100,9cyclictest10970-21ssh09:11:510
134889997,1cyclictest8411-21wget08:05:143
134889997,1cyclictest4052-21taskset09:02:373
134869998,1cyclictest14285-21ntp_states08:15:202
134829997,1cyclictest6177-21ls08:00:141
134829997,1cyclictest27044-21hwlatdetect07:35:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional