You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-15 - 19:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Aug 15, 2026 12:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3035122770,4sleep130353-21modprobe07:10:051
3035122770,4sleep130353-21modprobe07:10:051
2896922320,2sleep00-21swapper/007:05:190
2896922320,2sleep00-21swapper/007:05:190
306179915143,60cyclictest23588-21rm10:45:220
306179913634,60cyclictest0-21swapper/009:30:300
306179913038,65cyclictest0-21swapper/009:20:210
306189912932,73cyclictest0-21swapper/110:00:321
306189912932,73cyclictest0-21swapper/110:00:321
306179912535,63cyclictest0-21swapper/011:40:330
306179912138,61cyclictest19342-21ntpq10:00:320
306179912138,61cyclictest19342-21ntpq10:00:320
30617991074,80cyclictest7990-21unixbench_singl10:25:350
306179910728,58cyclictest0-21swapper/010:50:340
306179910129,55cyclictest16304-21sed10:35:340
306189910038,34cyclictest26955-21taskset09:30:311
30617999623,58cyclictest19621-21apt-get08:10:050
30617999531,11cyclictest10892-21ssh10:30:120
30617999427,50cyclictest29894-21ssh12:10:390
154422940,2sleep00-21swapper/012:33:220
30617999320,55cyclictest0-21swapper/009:55:110
30617999234,22cyclictest7950irq/56-dwc_otg_07:10:080
30617999126,47cyclictest22031-21fschecks_count08:15:230
3061799902,72cyclictest0-21swapper/009:50:180
30618998938,28cyclictest7940-21/usr/sbin/munin10:25:341
30617998924,52cyclictest21672-21latency_hist08:15:050
30617998921,58cyclictest0-21swapper/011:35:250
30617998921,44cyclictest0-21swapper/010:16:350
30617998824,54cyclictest0-21swapper/008:55:290
30618998735,29cyclictest18902-21ssh09:20:341
30617998728,42cyclictest3754-21aten_rbpower_vo07:25:200
30617998727,54cyclictest4478-21wget12:20:200
30617998724,50cyclictest12031-21sh12:29:400
30617998719,55cyclictest1842-21idleruntime-cro08:50:050
30617998539,8cyclictest30259-21apt-get09:35:170
30617998529,41cyclictest8095-21ntpq11:05:270
30617998520,54cyclictest16752-21ssh12:35:100
30618998453,25cyclictest23615-21/usr/sbin/munin10:45:241
30618998440,24cyclictest4737-21/usr/sbin/munin11:40:311
30617998426,43cyclictest17833-21idleruntime-cro08:05:050
30617998424,42cyclictest18366-21rm11:57:090
188622840,12sleep23061999cyclictest09:20:322
30617998325,38cyclictest22381-21/usr/sbin/munin09:25:250
30617998316,51cyclictest4726-21rm10:22:010
30617998224,39cyclictest25547-21ssh10:08:280
30617998223,56cyclictest0-21swapper/012:05:190
30617998220,42cyclictest0-21swapper/007:30:330
30617998124,42cyclictest11351-21rm09:11:370
30617998119,52cyclictest1395-21sh10:57:200
30617998119,52cyclictest1395-21sh10:57:200
30618998032,27cyclictest28527-21sshd10:50:351
30618998031,12cyclictest10221-21sh09:50:191
30617998022,46cyclictest26139-21ssh11:27:170
30617998022,46cyclictest26139-21ssh11:27:170
30617998021,49cyclictest24096-21if_eth008:20:250
30617998020,46cyclictest0-21swapper/011:50:210
30617997929,47cyclictest2980-21unixbench_singl07:20:340
30617997926,35cyclictest17681-21sort09:20:060
30617997926,35cyclictest17681-21sort09:20:060
30617997822,44cyclictest0-21swapper/011:11:180
30617997821,38cyclictest0-21swapper/008:50:270
30617997819,49cyclictest26726-21runrttasks08:26:000
30617997725,39cyclictest20230-21ssh12:00:100
30617997722,41cyclictest3702-21/usr/sbin/munin11:00:230
30617997722,38cyclictest28067-21aten_rbpower_cu11:30:200
30617997720,47cyclictest0-21swapper/009:46:270
30617997720,47cyclictest0-21swapper/009:46:270
30617997719,45cyclictest0-21swapper/011:16:240
3061999766,13cyclictest31-21ksoftirqd/211:40:332
30618997655,15cyclictest30616-21cyclictest07:10:081
30617997622,44cyclictest0-21swapper/008:34:020
30617997620,41cyclictest0-21swapper/009:45:050
30617997618,46cyclictest318-21fschecks_time07:15:230
30617997617,37cyclictest0-21swapper/010:40:280
294382760,8sleep328797-21/usr/sbin/munin07:05:263
294382760,8sleep328797-21/usr/sbin/munin07:05:263
138482760,3sleep229-21rcuc/211:51:252
30617997525,39cyclictest3502-21ssh12:19:080
30617997518,41cyclictest0-21swapper/011:24:540
200822750,2sleep20-21swapper/208:10:202
30617997422,39cyclictest0-21swapper/007:38:230
3061999736,31cyclictest28468-21ssh10:50:352
30619997248,17cyclictest23740-21ssh10:45:262
3061799727,61cyclictest0-21swapper/007:40:180
30617997220,40cyclictest0-21swapper/007:51:510
30617997220,40cyclictest0-21swapper/007:51:510
30617997219,42cyclictest32673-21latency_hist08:45:060
103062720,8sleep2301ktimersoftd/211:08:332
30617997120,43cyclictest0-21swapper/011:49:000
30617997119,37cyclictest0-21swapper/007:59:280
30617997023,30cyclictest28403-21ssh10:11:100
30617997014,43cyclictest0-21swapper/007:45:240
3061999697,36cyclictest7850irq/56-dwc_otg_10:25:352
30619996919,35cyclictest301ktimersoftd/209:30:322
196192690,5sleep30-21swapper/308:10:053
30620996814,22cyclictest19274-21ssh10:00:303
30620996814,22cyclictest19274-21ssh10:00:303
30617996817,42cyclictest0-21swapper/008:39:230
30617996817,42cyclictest0-21swapper/008:39:230
30619996712,14cyclictest301ktimersoftd/210:00:322
30619996712,14cyclictest301ktimersoftd/210:00:322
30618996626,19cyclictest32465-21cron08:45:051
30618996620,26cyclictest877-21ssh11:35:331
30620996521,18cyclictest23651-21cut10:45:253
30620996510,36cyclictest381ktimersoftd/310:25:353
30618996524,21cyclictest20492-21cat08:10:251
30618996524,12cyclictest22679-21/usr/sbin/munin09:25:281
30618996522,22cyclictest28455-21ssh11:30:251
30618996520,29cyclictest27686-21latency_hist11:30:061
30618996520,29cyclictest27686-21latency_hist11:30:061
30617996518,32cyclictest0-21swapper/009:03:330
30618996424,24cyclictest3671-21idleruntime-cro08:55:051
30617996420,31cyclictest0-21swapper/009:09:200
3061999622,35cyclictest31-21ksoftirqd/207:10:082
30618996227,20cyclictest13226-21ssh11:11:191
30618996227,19cyclictest2888-21ssh11:00:051
30618996227,19cyclictest2888-21ssh11:00:051
30618996226,20cyclictest8057-21ssh09:46:471
30618996226,20cyclictest8057-21ssh09:46:471
30618996225,18cyclictest11143-21ssh11:49:031
3061899616,26cyclictest0-21swapper/107:40:181
30618996125,12cyclictest22904-21ssh12:02:561
30618996123,11cyclictest27061-21ssh12:07:591
3062099604,33cyclictest7750irq/56-dwc_otg09:20:313
30618996027,20cyclictest20782-21ssh10:41:021
30618996024,24cyclictest26727-21runrttasks08:26:011
30618996022,20cyclictest13077-21ssh09:14:401
30618995928,7cyclictest12114-21ssh10:30:331
30618995928,15cyclictest1008-21ssh09:39:021
30618995927,13cyclictest12040-21ssh12:29:401
30618995925,21cyclictest21796-21ssh11:21:521
30618995925,15cyclictest29940-21ssh12:10:431
30618995923,12cyclictest1855-21rm12:16:091
30618995917,25cyclictest16468-21ssh10:35:411
3062099584,35cyclictest0-21swapper/309:50:193
3062099583,33cyclictest0-21swapper/310:50:333
30618995825,7cyclictest6413-21ssh12:21:551
30618995825,21cyclictest4732-21sh10:22:021
30618995823,20cyclictest14021-21taskset11:51:441
30618995823,15cyclictest3579-21cut11:00:221
30618995818,15cyclictest60-21kswapd009:55:111
300872582,49sleep20-21swapper/207:06:432
300872582,49sleep20-21swapper/207:06:432
3062099573,23cyclictest39-21ksoftirqd/309:30:313
3061999576,37cyclictest7750irq/56-dwc_otg09:50:192
30618995727,15cyclictest21765-21/usr/sbin/munin08:15:271
30618995725,20cyclictest14926-21seq12:32:251
30618995722,12cyclictest29901-21rm10:13:541
30618995722,10cyclictest0-21swapper/109:20:031
30618995722,10cyclictest0-21swapper/109:20:031
30618995721,20cyclictest3420-21ssh09:40:501
30618995628,14cyclictest13919-21sshd07:51:301
30618995628,14cyclictest13919-21sshd07:51:301
30618995618,14cyclictest17762-21cat12:35:331
3062099555,8cyclictest0-21swapper/311:40:333
30618995531,16cyclictest1829-21apt-get07:20:171
3061899552,6cyclictest7950irq/56-dwc_otg_08:00:171
30618995525,12cyclictest17616-21ssh11:16:481
30618995524,16cyclictest4808-21grep08:55:321
30618995523,12cyclictest8145-21ntp_states11:05:291
30618995519,23cyclictest0-21swapper/109:00:181
3061999547,30cyclictest0-21swapper/210:20:202
30618995427,14cyclictest15954-21sshd07:59:301
30618995422,19cyclictest1404-21taskset07:17:561
30618995422,17cyclictest5434-21/usr/sbin/munin07:30:331
30618995422,13cyclictest24888-21chrt08:20:411
30618995325,16cyclictest19319-21ssh11:58:541
30618995321,20cyclictest337-21ssh10:16:381
30618995317,22cyclictest24090-21rm10:05:471
30618995121,8cyclictest7850irq/56-dwc_otg_07:25:301
3062099482,8cyclictest7750irq/56-dwc_otg11:05:163
30618994821,17cyclictest12310-21runrttasks07:49:011
30618994815,10cyclictest0-21swapper/109:06:151
30618994719,13cyclictest28762-21seq08:34:011
3062099467,25cyclictest0-21swapper/308:55:193
3061999467,24cyclictest0-21swapper/208:05:192
30618994616,11cyclictest7850irq/56-dwc_otg_07:35:521
3061999457,25cyclictest0-21swapper/208:55:192
3061999456,25cyclictest0-21swapper/211:45:192
30619994520,7cyclictest31-21ksoftirqd/210:30:232
30618994516,10cyclictest0-21swapper/108:39:431
30618994516,10cyclictest0-21swapper/108:39:431
30618994422,10cyclictest1645-21chrt08:47:501
30618994414,15cyclictest0-21swapper/108:05:131
3062099437,23cyclictest0-21swapper/309:40:203
30620994324,9cyclictest7750irq/56-dwc_otg12:05:173
3061999436,23cyclictest1989-21ssh09:40:202
3061999431,8cyclictest23885-21rm11:25:162
3061999431,8cyclictest23885-21rm11:25:162
3062099427,23cyclictest0-21swapper/308:20:203
3061999427,22cyclictest0-21swapper/208:15:202
3061999421,5cyclictest7950irq/56-dwc_otg_11:55:162
3061999417,25cyclictest0-21swapper/208:45:192
3062099408,16cyclictest8482-21ssh12:25:163
3061999407,24cyclictest0-21swapper/208:20:202
3061999407,21cyclictest0-21swapper/207:25:192
30619994024,10cyclictest7750irq/56-dwc_otg12:10:282
243542400,1sleep30-21swapper/309:27:253
155242401,4sleep20-21swapper/209:16:302
155242401,4sleep20-21swapper/209:16:302
3061999397,25cyclictest0-21swapper/211:00:182
3061999393,3cyclictest7950irq/56-dwc_otg_09:25:202
3061999391,25cyclictest7950irq/56-dwc_otg_10:05:272
321462381,32sleep30-21swapper/309:36:363
3062099384,20cyclictest0-21swapper/307:10:083
30620993815,6cyclictest39-21ksoftirqd/309:55:063
3061999388,6cyclictest25818-21fschecks_time08:25:232
3061999387,17cyclictest7850irq/56-dwc_otg_11:35:202
3062099378,12cyclictest39-21ksoftirqd/311:00:063
3062099378,12cyclictest39-21ksoftirqd/311:00:063
3061999371,11cyclictest7950irq/56-dwc_otg_12:27:452
3062099367,5cyclictest7750irq/56-dwc_otg12:15:233
3062099366,5cyclictest37-21rcuc/308:05:053
30620993613,8cyclictest39-21ksoftirqd/311:35:283
3062099361,24cyclictest17912-21ls12:35:343
3062099361,20cyclictest21412-21timerwakeupswit12:00:333
30619993620,9cyclictest7750irq/56-dwc_otg12:35:222
3061999361,23cyclictest2329-21ls07:20:262
3061999361,16cyclictest7850irq/56-dwc_otg_10:16:312
3062099355,13cyclictest7850irq/56-dwc_otg_11:55:283
3062099353,6cyclictest7750irq/56-dwc_otg11:15:173
3062099353,4cyclictest7750irq/56-dwc_otg12:35:063
30620993515,5cyclictest31515-21sh12:13:373
3061999353,17cyclictest12166-21ssh11:10:272
30619993520,4cyclictest7750irq/56-dwc_otg09:45:172
30619993520,4cyclictest7750irq/56-dwc_otg09:45:172
3061999351,4cyclictest0-21swapper/212:35:062
30619993513,6cyclictest16566-21fschecks_time08:00:232
30619993513,4cyclictest3119-21latency_hist11:00:062
3062099346,9cyclictest7850irq/56-dwc_otg_07:25:233
3062099342,11cyclictest39-21ksoftirqd/308:50:053
30620993416,6cyclictest30745-21sh11:33:093
3061999346,4cyclictest7750irq/56-dwc_otg12:15:192
3061999342,15cyclictest0-21swapper/212:24:412
30619993411,12cyclictest0-21swapper/212:05:442
30619993410,17cyclictest2394-21ssh10:59:102
3062099335,18cyclictest7850irq/56-dwc_otg_10:41:023
3062099334,11cyclictest37-21rcuc/309:16:573
3062099334,11cyclictest37-21rcuc/309:16:573
30620993318,5cyclictest12287-21vmstat10:30:353
30620993315,5cyclictest37-21rcuc/307:30:303
30620993312,4cyclictest37-21rcuc/310:22:193
3061999338,14cyclictest0-21swapper/209:10:222
3061999337,12cyclictest7850irq/56-dwc_otg_10:40:212
3061999334,4cyclictest0-21swapper/211:33:112
3061999333,5cyclictest15744-21ssh10:35:272
3062099324,14cyclictest7850irq/56-dwc_otg_07:50:063
3062099323,7cyclictest7850irq/56-dwc_otg_10:35:343
30620993223,7cyclictest6368-21fschecks_count09:45:233
30620993223,7cyclictest6368-21fschecks_count09:45:233
3062099322,15cyclictest37-21rcuc/311:45:203
30620993212,6cyclictest25189-21runrttasks11:25:343
30620993212,6cyclictest25189-21runrttasks11:25:343
3062099321,18cyclictest7750irq/56-dwc_otg11:01:383
3061999327,4cyclictest7750irq/56-dwc_otg12:00:172
3061999322,10cyclictest11-21rcu_preempt09:35:172
30619993215,5cyclictest7006-21latency_hist07:35:062
3062099319,6cyclictest39-21ksoftirqd/308:15:203
3062099317,5cyclictest7850irq/56-dwc_otg_07:15:343
3062099315,4cyclictest7850irq/56-dwc_otg_12:20:233
3062099312,22cyclictest381ktimersoftd/309:14:433
30620993118,7cyclictest7750irq/56-dwc_otg11:24:323
3062099311,20cyclictest32481-21latency_hist08:45:053
30620993111,4cyclictest0-21swapper/310:12:053
3061999318,3cyclictest0-21swapper/211:23:102
3061999317,5cyclictest7850irq/56-dwc_otg_08:40:202
3061999316,6cyclictest7850irq/56-dwc_otg_07:50:232
3061999316,6cyclictest7850irq/56-dwc_otg_07:50:232
3061999315,4cyclictest7750irq/56-dwc_otg10:10:332
3061999313,6cyclictest19456-21cat11:20:052
3061999312,14cyclictest7850irq/56-dwc_otg_08:35:322
3061999312,14cyclictest7850irq/56-dwc_otg_08:35:322
3061999311,6cyclictest14672-21cat09:55:232
30619993110,6cyclictest0-21swapper/208:50:202
30619993110,5cyclictest8032-21latency09:05:252
3062099303,5cyclictest30932-21aten_rbpower_cu10:15:193
3062099302,7cyclictest15605-21sed07:55:333
30620993024,4cyclictest39-21ksoftirqd/311:50:223
3062099302,19cyclictest0-21swapper/308:34:503
3062099302,17cyclictest381ktimersoftd/310:06:033
30620993010,11cyclictest37-21rcuc/307:40:583
3061999306,6cyclictest7850irq/56-dwc_otg_09:05:052
30619993025,3cyclictest0-21swapper/207:50:062
3062099298,11cyclictest0-21swapper/309:01:353
3062099296,14cyclictest0-21swapper/311:10:203
3062099294,5cyclictest27057-21cron08:30:053
30620992916,7cyclictest7750irq/56-dwc_otg09:08:553
3062099291,24cyclictest30672-21latency_hist08:40:053
3062099291,24cyclictest30672-21latency_hist08:40:053
3062099291,17cyclictest39-21ksoftirqd/308:10:233
3061999296,5cyclictest7850irq/56-dwc_otg_07:20:062
3062099284,8cyclictest39-21ksoftirqd/308:50:233
3062099282,15cyclictest0-21swapper/307:36:463
3062099281,7cyclictest7850irq/56-dwc_otg_07:20:343
30619992814,4cyclictest9347-21cut07:40:222
3061999281,24cyclictest0-21swapper/207:55:182
3062099278,10cyclictest37-21rcuc/307:54:093
3062099278,10cyclictest37-21rcuc/307:54:093
3061999274,10cyclictest7850irq/56-dwc_otg_08:30:202
3061999273,6cyclictest7950irq/56-dwc_otg_07:35:282
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional