You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 08:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sat Jul 18, 2026 00:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14331216596,62sleep10-21swapper/119:08:451
141042140105,24sleep20-21swapper/219:05:492
142412139103,25sleep30-21swapper/319:07:353
143202138102,25sleep00-21swapper/019:08:350
68592620,2sleep10-21swapper/121:13:301
308902620,1sleep10-21swapper/119:45:231
308902620,1sleep10-21swapper/119:45:231
165212620,1sleep10-21swapper/122:55:201
165212620,1sleep10-21swapper/122:55:201
316812570,1sleep00-21swapper/023:05:540
258322570,1sleep30-21swapper/322:15:013
146222560,2sleep10-21swapper/123:40:431
283292530,2sleep10-21swapper/120:55:171
242692480,1sleep00-21swapper/023:00:240
14702994811,27cyclictest22087-21sh00:35:042
1470299480,32cyclictest18921-21ssh22:33:292
252782470,1sleep00-21swapper/023:25:210
165362470,2sleep30-21swapper/300:30:213
1470299471,32cyclictest26192-21ssh23:25:382
1470299461,34cyclictest6008-21sh22:47:362
14702994611,26cyclictest20233-21sh21:23:132
14702994611,26cyclictest20233-21sh21:23:122
1470299451,27cyclictest20579-21sh00:33:482
1470299451,26cyclictest10623-21chrt22:50:272
273442440,2sleep30-21swapper/323:03:213
174802440,1sleep00-21swapper/022:08:440
1470299441,33cyclictest3743-21sh23:10:082
1470299441,33cyclictest18264-21sh23:44:202
1470299441,29cyclictest30384-21ssh23:52:562
1470299441,29cyclictest28571-21ssh21:52:532
1470299441,26cyclictest7538-21sh00:24:182
1470299441,26cyclictest31626-21taskset21:02:412
1470299441,26cyclictest23360-21sh23:24:342
1470299440,34cyclictest31229-21sh22:42:132
1470199448,25cyclictest17800-21sh21:20:491
1470199448,25cyclictest17800-21sh21:20:481
1470199441,29cyclictest25684-21seq23:25:251
1470199440,20cyclictest8363-21ntp_kernel_pll_00:00:231
16062430,1sleep30-21swapper/319:55:123
1470299431,27cyclictest6353-21ssh22:24:312
1470299431,26cyclictest1641-21sh21:56:352
1470299431,26cyclictest1550-21diskmemload23:03:582
1470299431,24cyclictest1310-21taskset21:07:182
1470299430,33cyclictest10644-21sh22:27:032
1470299430,31cyclictest29127-21ssh21:30:022
1470299430,28cyclictest6859-21ssh21:36:382
1470299430,26cyclictest21001-21chrt20:39:052
1470299430,26cyclictest15156-21ssh21:19:582
1470299430,26cyclictest13664-21sh21:41:372
1470299430,25cyclictest30741-21chrt22:18:212
1470199431,22cyclictest12656-21sh21:40:371
14703994217,22cyclictest1554-21ssh23:08:113
1470299421,25cyclictest23790-21sh00:11:412
1470299421,25cyclictest19796-21sh00:09:262
1470299421,25cyclictest10824-21chrt00:25:502
1470299421,24cyclictest31603-21ssh23:05:502
1470299421,24cyclictest22673-21sh22:35:292
1470299421,24cyclictest14486-21chrt20:22:522
1470299421,23cyclictest1349-21taskset19:54:592
14702994210,20cyclictest30101-21sh00:16:152
1470299420,32cyclictest15134-21sh23:18:092
1470299420,26cyclictest2356-21ssh23:55:322
1470299420,25cyclictest29377-21chrt19:43:282
1470299420,25cyclictest23088-21sh23:47:242
1470299420,25cyclictest11816-21ssh22:04:522
1470299420,24cyclictest16275-21taskset19:10:382
1470299420,24cyclictest1550-21diskmemload21:25:492
1470299420,24cyclictest11934-21chrt20:15:372
14701994211,22cyclictest31527-21sh21:30:561
1470299411,30cyclictest8789-21sh23:36:422
1470299411,30cyclictest21775-21sh22:59:462
1470299411,30cyclictest21775-21sh22:59:462
1470299411,26cyclictest9536-21ssh00:00:562
1470299411,26cyclictest31522-21taskset19:48:332
1470299411,26cyclictest31522-21taskset19:48:332
1470299411,23cyclictest22909-21chrt20:40:492
1470299410,31cyclictest4757-21ssh21:11:272
1470299410,28cyclictest19166-21ssh22:10:102
1470299410,28cyclictest19166-21ssh22:10:092
1470299410,27cyclictest9855-21taskset20:11:262
1470299410,25cyclictest16152-21ssh22:07:252
1470199411,30cyclictest29370-21sh23:51:551
1470199410,1cyclictest0-21swapper/122:10:061
1470199410,1cyclictest0-21swapper/122:10:051
237452400,1sleep00-21swapper/023:24:570
1470299401,23cyclictest20535-21sh21:46:402
1470299400,26cyclictest31937-21ssh23:30:172
1470299400,26cyclictest27093-21chrt19:36:492
1470299400,26cyclictest23463-21chrt20:45:032
1470299400,24cyclictest7525-21chrt20:06:352
1470299400,23cyclictest16564-21taskset20:27:022
1470199405,19cyclictest20329-21sh21:46:281
1470199401,29cyclictest32016-21sh23:06:131
1470199401,24cyclictest9543-21sh23:14:221
1470199401,22cyclictest26889-21sh22:39:391
1470199401,22cyclictest22294-21sh23:46:371
1470199401,21cyclictest24522-21chrt21:25:431
1470299391,25cyclictest29369-21chrt20:56:272
1470299391,24cyclictest3156-21chrt19:55:402
1470299391,24cyclictest22793-21taskset19:26:332
1470299391,24cyclictest18548-21chrt19:17:162
1470299390,25cyclictest25175-21chrt19:34:412
1470299390,23cyclictest27328-21taskset20:52:382
1470299390,22cyclictest18727-21chrt20:32:192
1470299390,22cyclictest18727-21chrt20:32:182
1470199398,15cyclictest6323-21sh23:59:311
1470199391,24cyclictest30107-21sh22:17:431
1470199391,22cyclictest26660-21sh00:14:321
1470199391,22cyclictest1647-21sh23:31:311
1470199391,21cyclictest11141-21sh23:39:021
1470199390,21cyclictest6046-21sh22:00:151
1470199390,21cyclictest5149-21sh22:46:451
1470199381,22cyclictest1550-21diskmemload00:29:071
1470199381,21cyclictest28825-21sh23:04:481
1470199381,21cyclictest14408-21sh22:05:431
1470199381,20cyclictest20918-21taskset20:38:001
1470199381,20cyclictest11737-21sh22:51:341
1470199380,22cyclictest24651-21sh00:36:061
1470199380,21cyclictest10407-21sh22:26:481
1470199380,20cyclictest5834-21sh21:35:371
181722370,4sleep018170-21hddtemp_smartct21:45:160
1470299371,21cyclictest19015-21taskset19:20:052
1470199371,20cyclictest17097-21sh22:31:421
1470199371,20cyclictest11447-21sh21:16:201
1470199371,19cyclictest20257-21sh00:33:281
1470199370,21cyclictest4933-21sh21:59:491
1470199370,21cyclictest30629-21sh21:54:551
1470199370,21cyclictest30182-21sh00:16:201
1470199370,21cyclictest2327-21sh22:20:331
1470199370,20cyclictest3866-21sh00:20:381
1470199370,20cyclictest14836-21sh23:17:501
239352360,2sleep30-21swapper/323:48:153
1470199361,19cyclictest23002-21taskset20:42:011
1470199361,19cyclictest19999-21sh23:21:141
1470199360,20cyclictest29579-21sh22:40:341
192692350,2chrt0-21swapper/023:45:060
1470399348,6cyclictest26909-21sed20:50:253
1470199340,20cyclictest12273-21chrt20:19:581
1470199340,19cyclictest22857-21taskset19:27:261
1470399333,5cyclictest12589-21apt-get23:40:123
14703993311,4cyclictest31209-21kworker/u8:122:22:103
14703993310,3cyclictest4074-21cstates21:35:123
1470299331,20cyclictest5513-21taskset20:03:142
1470199331,19cyclictest20853-21taskset19:24:081
1470199330,18cyclictest0-21swapper/100:05:421
1470399325,8cyclictest8831-21cpuspeed_turbos00:25:113
1470399322,7cyclictest24868-21fschecks_time21:50:173
1470399322,6cyclictest10724-21ssh21:40:123
1470399321,9cyclictest2684-21kernelversion00:20:183
14703993212,4cyclictest1550-21diskmemload00:06:453
14703993210,5cyclictest9030-21expr21:15:143
14703993210,4cyclictest32563-21switchtime21:55:253
14703993210,4cyclictest22471-21/usr/sbin/munin21:25:143
14703993210,4cyclictest14661-21sed22:30:153
1470199321,19cyclictest3180-21taskset19:55:591
1470399319,5cyclictest14876-21sh22:54:393
1470399319,4cyclictest21032-21cpuspeed_turbos22:35:113
1470399318,4cyclictest4208-21sh23:10:153
1470399317,5cyclictest13373-21ntp_kernel_err22:05:183
1470399317,4cyclictest0-21swapper/300:15:143
1470399312,8cyclictest15645-21apt-get21:20:123
1470399312,8cyclictest15645-21apt-get21:20:123
1470399311,9cyclictest560-21dbus-daemon20:30:003
1470399311,9cyclictest22294-21latency_hist23:00:003
1470399311,9cyclictest22294-21latency_hist23:00:003
1470399311,9cyclictest18489-21grep23:20:203
1470399311,7cyclictest22138-21ssh00:10:233
14703993111,3cyclictest24994-21sh22:14:253
14703993111,3cyclictest24994-21sh22:14:253
1470399311,11cyclictest2035-21/usr/sbin/munin22:45:263
14701993111,17cyclictest23390-21cat19:30:131
1470399307,4cyclictest2115-21cpuspeed_turbos21:10:133
1470399305,5cyclictest17044-21dump-pmu-power23:20:003
1470399303,4cyclictest0-21swapper/323:35:123
1470399301,8cyclictest32127-21ls23:30:193
1470399301,8cyclictest29667-21idleruntime-cro21:00:003
14703993011,4cyclictest1550-21diskmemload23:28:273
14703993010,4cyclictest28898-21sh23:51:283
14703993010,3cyclictest1214-21rm21:33:013
1470199300,1cyclictest0-21swapper/119:10:211
1470399299,5cyclictest24103-21cat20:45:163
1470399299,3cyclictest27349-21latency_hist19:40:003
1470399296,4cyclictest0-21swapper/320:35:143
1470399295,6cyclictest18298-21unixbench_multi19:15:273
1470399291,9cyclictest7739-21fschecks_count00:00:163
1470399291,9cyclictest6347-21fschecks_time20:05:153
1470399291,9cyclictest6040-21tune2fs22:00:163
1470399291,8cyclictest21718-21cut19:25:153
14703992910,4cyclictest9504-21timerandwakeup20:10:233
1470399290,5cyclictest24003-21cat21:50:013
1470399286,4cyclictest21829-21grep20:40:163
1470399285,5cyclictest17495-21fschecks_time20:30:143
1470399285,5cyclictest17495-21fschecks_time20:30:143
1470399281,8cyclictest32401-21gltestperf19:50:163
1470399281,8cyclictest29117-21grep19:40:243
1470399281,8cyclictest1656-21ssh21:10:003
1470399281,10cyclictest22230-21apt-get00:35:133
1470399280,9cyclictest31389-21grep21:00:263
1470199284,21cyclictest698-21sed19:50:231
1470199284,21cyclictest25816-21fschecks_count19:35:141
14701992813,13cyclictest5665-21latency_hist20:05:001
14701992811,15cyclictest17965-21gzip19:15:241
14701992810,15cyclictest31811-21cron21:05:001
91832270,4chrt0-21swapper/021:15:150
1470399272,8cyclictest27759-21ssh22:40:123
1470399271,9cyclictest31138-21chrt19:45:253
1470399271,9cyclictest31138-21chrt19:45:253
1470399271,9cyclictest16641-21awk19:15:013
1470399271,6cyclictest8118-21ssh22:25:173
1470199275,20cyclictest6695-21cat20:05:181
1470199271,23cyclictest17170-21tr20:30:121
1470199271,23cyclictest17170-21tr20:30:111
1470199271,23cyclictest1654-21ls21:10:001
1470199271,22cyclictest14970-21/usr/sbin/munin20:25:141
1470199271,21cyclictest23499-21apt-get20:45:121
1470399262,4cyclictest0-21swapper/320:00:233
1470399261,8cyclictest20950-21awk19:25:013
14703992610,4cyclictest1550-21diskmemload23:59:513
1470199265,18cyclictest27758-21cpuspeed_turbos19:40:121
1470099261,1cyclictest0-21swapper/019:13:030
14700992610,4cyclictest66050lldpd23:51:580
1470399251,8cyclictest14656-21cron20:25:003
1470199255,17cyclictest25929-21cstates20:50:151
1470199251,21cyclictest8430-21cstates20:10:111
1470199250,1cyclictest0-21swapper/120:20:131
316072230,9sleep012-21ksoftirqd/022:42:350
1470099237,11cyclictest2429-21kworker/u8:200:00:130
14700992213,7cyclictest17713-21ls19:15:200
14700992112,7cyclictest26314-21seq00:14:130
14700992112,6cyclictest28804-21cat00:40:010
14700992112,5cyclictest0-21swapper/000:30:110
14700992110,7cyclictest5429-21cat23:35:000
1470099209,8cyclictest26390-21df_inode22:15:140
1470099202,6cyclictest434-21apt-get23:55:110
1470099202,13cyclictest5612-21ssh23:35:030
1470099202,12cyclictest7283-21ssh22:25:120
14700992014,4cyclictest53050irq/142-enp3s0-22:20:220
14700992014,4cyclictest19610-21hwlatdetect19:20:160
14700992012,6cyclictest3478-21seq19:59:500
14700992012,6cyclictest32478-21idleruntime19:50:160
14700992012,6cyclictest31700-21tune2fs21:55:160
14700992012,6cyclictest30928-21sed21:55:000
14700992012,6cyclictest21155-21seq19:25:080
14700992011,7cyclictest12240-21grep23:15:240
1470099200,5cyclictest0-21swapper/022:13:340
1470099200,5cyclictest0-21swapper/022:13:340
1470399199,7cyclictest25225-21basename19:34:593
1470099199,7cyclictest660-21lldpd20:45:130
1470099198,3cyclictest14241-21cstates00:05:130
14700991913,4cyclictest12763-21sh22:52:350
14700991912,5cyclictest9000-21sed20:10:170
14700991912,5cyclictest4548-21basename21:35:160
14700991912,5cyclictest32218-21cut21:05:130
14700991912,5cyclictest28783-21seq21:29:550
14700991912,5cyclictest13925-21python323:40:230
14700991911,6cyclictest9296-21cat00:25:150
14700991911,6cyclictest5417-21grep23:10:260
14700991911,6cyclictest22024-21ls23:00:000
14700991911,6cyclictest22024-21ls22:59:590
14700991911,6cyclictest18033-21seq20:30:200
14700991911,6cyclictest18033-21seq20:30:200
14700991910,6cyclictest6650-21ntp_states22:00:220
14700991910,6cyclictest30712-21pluginstate19:45:220
14700991910,6cyclictest30712-21pluginstate19:45:210
1470099182,12cyclictest1980-21apt-get21:10:130
1470099182,11cyclictest29287-21apt-get21:30:130
14700991813,3cyclictest28945-21swap20:55:240
14700991813,3cyclictest28338-21sh00:15:190
14700991812,4cyclictest5773-21sh00:22:330
14700991812,4cyclictest17566-21sh22:32:090
14700991811,5cyclictest7075-21swap20:05:230
14700991811,5cyclictest5422-21seq20:02:040
1470099181,11cyclictest20880-21ssh22:35:110
14700991810,6cyclictest22112-21sed21:25:000
14700991810,6cyclictest22112-21sed21:25:000
14700991810,6cyclictest13988-21cat20:20:230
1470399178,6cyclictest8213-21packagekitd20:15:073
1470099177,7cyclictest14904-21cat20:25:120
1470099171,5cyclictest2261-21cstates22:45:120
14700991710,5cyclictest30816-21ntpq21:00:200
14700991710,5cyclictest23646-21seq19:30:140
14700991710,5cyclictest20988-21seq20:38:550
14700991710,5cyclictest12-21ksoftirqd/019:40:000
14700991710,4cyclictest1550-21diskmemload21:40:380
1470099168,6cyclictest25960-21df20:50:150
1470099168,5cyclictest27599-21acpi19:40:100
1470099158,5cyclictest10829-21cut20:15:160
14700991510,3cyclictest22670-21timerwakeupswit20:40:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional