You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 19:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Apr 06, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1447499792,54cyclictest22908-21ssh10:35:132
1445599593,5cyclictest131rcu_preempt09:45:120
1446399475,6cyclictest131rcu_preempt08:15:121
14482994621,17cyclictest28366irq/58-eth010:00:143
14474994618,23cyclictest27-21ksoftirqd/211:40:122
1445599441,31cyclictest31913-21ssh10:40:140
1447499432,28cyclictest8172-21ssh12:15:152
1447499432,14cyclictest924-21snmpd12:10:132
1447499422,27cyclictest25098-21ssh10:00:142
1448299414,22cyclictest28366irq/58-eth010:50:133
1448299402,26cyclictest28366irq/58-eth011:35:123
1447499394,20cyclictest0-21swapper/209:10:122
1447499391,23cyclictest1535-21ssh10:05:142
14463993922,11cyclictest0-21swapper/109:50:121
1448299372,21cyclictest28366irq/58-eth012:05:143
1446399371,31cyclictest11865-21ssh11:05:121
14455993729,5cyclictest13879-21sh10:30:120
1447499361,30cyclictest8414-21ssh10:45:162
1447499361,30cyclictest8414-21ssh10:45:162
14482993521,11cyclictest28366irq/58-eth009:30:123
1448299351,21cyclictest28366irq/58-eth010:25:133
1446399351,28cyclictest13880-21ssh10:30:121
1445599358,20cyclictest12-21ksoftirqd/009:05:150
14482993427,4cyclictest28366irq/58-eth009:35:133
1448299342,26cyclictest28366irq/58-eth011:50:123
1448299342,26cyclictest28366irq/58-eth011:50:113
1448299341,19cyclictest28366irq/58-eth011:05:133
1446399346,12cyclictest21-21ksoftirqd/108:55:141
1446399342,23cyclictest18424-21ssh09:20:141
1446399342,21cyclictest21-21ksoftirqd/107:35:121
1446399342,21cyclictest21-21ksoftirqd/107:35:121
1448299335,18cyclictest0-21swapper/309:55:133
1448299333,18cyclictest28366irq/58-eth009:15:123
1448299331,21cyclictest28366irq/58-eth011:15:123
1447499337,21cyclictest26155-21apt-get12:25:142
1445599332,15cyclictest32748-21ssh09:26:100
1448299328,12cyclictest0-21swapper/308:00:123
14482993212,11cyclictest28366irq/58-eth009:25:133
1445599321,21cyclictest26503-1kworker/u9:210:25:130
14463993118,7cyclictest31246-21kworker/1:208:40:131
1447499301,22cyclictest3905-21ssh09:30:132
1445599308,11cyclictest12-21ksoftirqd/012:30:110
1448299295,16cyclictest28366irq/58-eth010:40:133
1448299293,19cyclictest0-21swapper/311:20:123
14482992916,9cyclictest28366irq/58-eth010:30:133
1447499293,20cyclictest0-21swapper/212:24:112
1447499293,18cyclictest22346irq/40-dwc2_hso09:15:132
1447499293,16cyclictest15584-21ssh11:25:132
14474992914,10cyclictest2774-21apt-get12:30:122
14463992925,2cyclictest0-21swapper/111:00:131
14463992923,3cyclictest21-21ksoftirqd/107:50:131
14463992915,9cyclictest21-21ksoftirqd/110:15:151
1445599292,19cyclictest12727-21ssh09:35:100
1447499282,18cyclictest22346irq/40-dwc2_hso08:50:122
14474992811,10cyclictest0-21swapper/209:25:142
1446399284,20cyclictest16132-21ssh09:55:131
1446399283,17cyclictest26345-21ssh10:55:141
1446399281,20cyclictest26166-21ssh12:25:131
14455992819,4cyclictest0-21swapper/010:55:140
1447499276,9cyclictest27-21ksoftirqd/212:36:062
1447499273,16cyclictest22346irq/40-dwc2_hso07:50:122
1447499272,19cyclictest0-21swapper/207:10:132
1447499271,20cyclictest32479-21diskmemload09:40:112
1448299260,21cyclictest28366irq/58-eth011:55:133
1447499265,8cyclictest27-21ksoftirqd/212:03:092
1447499263,15cyclictest22346irq/40-dwc2_hso11:20:122
14474992617,5cyclictest0-21swapper/209:45:132
14474992611,12cyclictest0-21swapper/209:00:232
14474992610,10cyclictest27-21ksoftirqd/211:45:132
14463992620,3cyclictest21-21ksoftirqd/111:45:131
14463992613,10cyclictest21-21ksoftirqd/110:20:121
1445599261,20cyclictest15600-21ssh11:25:140
1445599261,19cyclictest28154-21ssh11:50:130
1445599261,19cyclictest28154-21ssh11:50:130
1445599261,12cyclictest2490-21cron08:05:010
14482992512,8cyclictest28366irq/58-eth009:40:113
1447499259,3cyclictest13869-21apt-get10:30:142
1447499258,8cyclictest27-21ksoftirqd/211:51:502
1447499258,8cyclictest27-21ksoftirqd/211:51:502
1447499253,14cyclictest22346irq/40-dwc2_hso08:45:142
1447499252,8cyclictest17704-21ssh09:35:372
1447499252,8cyclictest14207-21ssh09:53:102
1446399253,18cyclictest401-21ssh09:10:121
1446399252,20cyclictest25084-21ssh10:00:131
1445599254,15cyclictest0-21swapper/008:05:130
1445599254,15cyclictest0-21swapper/008:05:130
1445599253,16cyclictest11837-21ssh12:35:150
1445599251,20cyclictest31612-21ssh12:10:130
1448299243,13cyclictest28466irq/59-eth012:25:133
1448299241,15cyclictest28366irq/58-eth009:10:123
1447499248,11cyclictest27-21ksoftirqd/211:07:192
1447499247,11cyclictest27-21ksoftirqd/207:25:322
1447499246,8cyclictest27-21ksoftirqd/210:22:442
1447499245,8cyclictest27-21ksoftirqd/208:00:302
1447499243,8cyclictest27-21ksoftirqd/210:12:592
1447499243,4cyclictest27-21ksoftirqd/208:05:252
1447499243,4cyclictest27-21ksoftirqd/208:05:252
1447499242,16cyclictest22346irq/40-dwc2_hso11:00:132
14474992411,4cyclictest27-21ksoftirqd/212:08:382
1446399242,18cyclictest4941-21ssh10:25:141
1446399242,18cyclictest10238-21ssh11:40:121
1446399242,17cyclictest11838-21sh12:35:151
1445599241,18cyclictest28371-21ssh10:20:130
1445599241,17cyclictest4837-21ssh11:35:240
1448299231,14cyclictest0-21swapper/310:20:133
1447499238,4cyclictest27-21ksoftirqd/210:27:392
1447499236,12cyclictest27-21ksoftirqd/211:10:152
1447499235,11cyclictest27-21ksoftirqd/210:53:512
1447499234,8cyclictest27-21ksoftirqd/207:35:252
1447499234,8cyclictest27-21ksoftirqd/207:35:242
1447499233,7cyclictest0-21swapper/207:40:122
1447499233,13cyclictest27-21ksoftirqd/211:34:102
1447499232,6cyclictest0-21swapper/209:58:102
1447499232,14cyclictest27-21ksoftirqd/211:18:272
14474992311,9cyclictest0-21swapper/208:20:172
14474992310,6cyclictest4247-21cron11:55:012
1446399231,18cyclictest21754-21sh09:40:111
1445599234,13cyclictest29981-21ssh11:15:120
1445599232,16cyclictest22098-21ssh09:20:260
14455992314,7cyclictest20588-21cron11:10:010
14455992312,8cyclictest24656-21cron10:00:010
1445599231,17cyclictest10447-21ssh10:10:130
1448299229,6cyclictest0-21swapper/311:30:313
1448299224,12cyclictest0-21swapper/312:15:153
1448299222,13cyclictest0-21swapper/310:15:153
1447499224,8cyclictest27-21ksoftirqd/210:15:152
1447499224,10cyclictest131rcu_preempt11:38:182
1447499222,10cyclictest31684-21timerwakeupswit09:05:302
1447499222,10cyclictest30106-21proc_pri10:55:282
1447499221,18cyclictest19429-21find07:15:112
1447499221,17cyclictest18462-21tune2fs08:55:172
14474992211,4cyclictest27-21ksoftirqd/210:44:122
1447499221,11cyclictest924-21snmpd08:35:122
1445599223,15cyclictest0-21swapper/009:50:130
1445599222,18cyclictest23067-21ls11:45:280
1445599222,16cyclictest13684-21ssh12:00:140
1445599221,16cyclictest8821-21uniq08:45:160
1448299212,16cyclictest28366irq/58-eth007:20:113
14482992112,6cyclictest28366irq/58-eth012:35:143
14482992112,6cyclictest28366irq/58-eth012:10:133
14482992112,6cyclictest28366irq/58-eth010:05:133
1448299211,15cyclictest0-21swapper/307:50:163
1447499215,7cyclictest27-21ksoftirqd/207:20:242
1447499214,4cyclictest131rcu_preempt09:20:152
1447499213,13cyclictest22346irq/40-dwc2_hso08:10:132
14474992112,7cyclictest27-21ksoftirqd/207:45:302
1446399219,3cyclictest131rcu_preempt12:00:291
1446399213,15cyclictest24652-21ssh11:30:141
1446399213,15cyclictest21092-21ssh11:10:151
14463992114,4cyclictest4341-21kthreadcore10:05:221
14463992113,6cyclictest20634-21cat12:20:231
1446399211,18cyclictest17030-21ls10:50:011
1446399211,18cyclictest17030-21ls10:50:011
1446399211,17cyclictest6578-21ssh11:20:131
1446399211,15cyclictest15758-21cut11:25:161
14463992110,7cyclictest3955-21awk09:30:131
14455992114,5cyclictest0-21swapper/009:10:160
1445599211,16cyclictest22608-21ssh12:05:140
1445599210,18cyclictest28124-21ls10:20:020
1448299207,6cyclictest0-21swapper/311:45:123
1448299203,11cyclictest0-21swapper/312:00:103
14482992011,6cyclictest28366irq/58-eth012:20:143
1447499208,8cyclictest1403-21apt-get07:30:122
1447499207,10cyclictest26894-21fschecks_time08:30:162
1447499207,10cyclictest26894-21fschecks_time08:30:162
1447499202,9cyclictest22088-21fschecks_time08:25:162
1447499201,9cyclictest15916-21switchtime08:15:272
1447499201,4cyclictest27-21ksoftirqd/207:55:142
14474992011,7cyclictest0-21swapper/208:40:162
1446399202,16cyclictest27378-21ssh09:25:131
1446399201,13cyclictest17999-21latency_hist09:20:021
14455992014,4cyclictest0-21swapper/008:25:270
14455992013,5cyclictest0-21swapper/008:50:250
14455992012,5cyclictest0-21swapper/007:30:150
1445599200,14cyclictest23039-21ls09:00:010
1448299199,8cyclictest0-21swapper/310:56:383
1448299199,6cyclictest0-21swapper/311:01:213
1448299197,7cyclictest0-21swapper/310:35:143
1448299192,9cyclictest0-21swapper/311:40:133
1448299192,11cyclictest0-21swapper/308:50:133
1448299191,9cyclictest28366irq/58-eth008:15:123
14482991912,5cyclictest28366irq/58-eth011:10:143
14482991911,5cyclictest28366irq/58-eth010:45:163
14482991911,5cyclictest28366irq/58-eth010:45:163
14482991911,3cyclictest0-21swapper/309:50:133
1448299191,13cyclictest0-21swapper/309:50:023
1448299191,13cyclictest0-21swapper/308:30:273
1448299191,13cyclictest0-21swapper/308:30:273
1448299191,13cyclictest0-21swapper/307:15:153
1446399193,10cyclictest0-21swapper/109:35:101
1446399192,13cyclictest26289-21ntp_states12:05:261
1446399191,15cyclictest32279-21ls07:25:221
1446399191,15cyclictest30105-21ssh11:15:151
1446399191,15cyclictest1463-21ls11:35:161
1446399191,14cyclictest0-21swapper/107:15:131
1446399191,13cyclictest3096-21ssh10:40:261
1445599199,8cyclictest26298-21diskstats12:25:160
1445599198,8cyclictest29599-21cron11:15:010
1445599192,10cyclictest0-21swapper/008:30:120
1445599192,10cyclictest0-21swapper/008:30:120
1445599191,7cyclictest4000-21df_inode09:30:140
14455991911,6cyclictest4982-21wc10:05:230
14455991911,6cyclictest20810-21ntp_kernel_pll_12:20:260
14455991911,5cyclictest0-21swapper/008:15:180
14455991911,5cyclictest0-21swapper/008:10:220
1445599191,10cyclictest12476-21tr11:40:200
14455991910,6cyclictest0-21swapper/007:45:270
14455991910,6cyclictest0-21swapper/007:20:160
1448299183,11cyclictest0-21swapper/311:25:133
1448299182,10cyclictest0-21swapper/312:30:053
1448299182,10cyclictest0-21swapper/307:45:143
1448299181,13cyclictest0-21swapper/308:20:273
1448299181,13cyclictest0-21swapper/307:10:273
1448299181,12cyclictest0-21swapper/309:00:273
1448299181,12cyclictest0-21swapper/308:40:283
14482991810,3cyclictest0-21swapper/309:21:273
1447499181,8cyclictest14484-21latency_hist07:10:012
1446399186,8cyclictest21-21ksoftirqd/108:20:121
1446399182,5cyclictest53146usb-storage07:30:121
1446399181,5cyclictest31587-21apt-get12:10:131
1446399181,14cyclictest30840-21kthreadcore09:05:231
1446399181,13cyclictest30608-21ssh09:45:131
1446399181,13cyclictest30033-21apt-get08:00:031
1446399181,13cyclictest21888-21find08:25:121
1446399181,10cyclictest22891-21apt-get10:35:141
14463991810,6cyclictest10378-21sshd10:10:101
14463991810,5cyclictest22032-21usage_EthcTask10:50:331
1445599188,7cyclictest20930-21unixbench_multi08:20:270
1445599188,7cyclictest17371-21ssh10:50:140
1445599186,10cyclictest30132-21apt-config08:00:030
14455991812,4cyclictest0-21swapper/007:40:310
14455991811,5cyclictest0-21swapper/008:40:230
14455991811,4cyclictest8389-21acpi10:45:150
14455991811,4cyclictest8389-21acpi10:45:150
1445599181,14cyclictest6220-21cpuspeed07:35:140
1445599181,14cyclictest6220-21cpuspeed07:35:140
14455991811,4cyclictest22480-21kthreadcore09:40:170
1445599181,14cyclictest13099-21ssh09:15:260
14455991811,4cyclictest0-21swapper/011:55:280
14455991810,6cyclictest171-21systemd-network07:10:130
14455991810,6cyclictest0-21swapper/009:00:270
1445599180,12cyclictest28655-21timerwakeupswit11:30:290
1448299178,6cyclictest28366irq/58-eth010:10:123
1448299172,10cyclictest0-21swapper/308:45:143
1448299172,10cyclictest0-21swapper/308:35:123
1448299172,10cyclictest0-21swapper/308:25:143
1448299172,10cyclictest0-21swapper/307:40:123
1446399176,3cyclictest131rcu_preempt12:15:241
1446399173,11cyclictest17029-21kthreadcore07:45:191
1446399173,10cyclictest10930-21ntp_kernel_pll_08:10:241
1446399172,11cyclictest14855-21grep07:40:291
1446399171,13cyclictest5325-21irqrtprio11:55:191
1446399171,13cyclictest1027-21kthreadcore08:35:191
1446399171,12cyclictest2906-21diskstats12:30:161
1446399170,8cyclictest28339-21fschecks_time11:50:151
1446399170,8cyclictest28339-21fschecks_time11:50:151
1445599179,5cyclictest3634-21kthreadcore11:00:200
1445599172,13cyclictest24358-21find07:50:270
1445599171,7cyclictest26461-21kthreadcore10:00:210
1445599171,11cyclictest839-21munin-node12:15:150
14455991710,4cyclictest22907-21sh10:35:130
1448299162,9cyclictest0-21swapper/307:55:133
1448299161,9cyclictest0-21swapper/309:05:303
1446399168,6cyclictest24603-21irqcore07:20:171
1446399168,6cyclictest15850-21cut07:10:201
1446399163,10cyclictest14619-21cat07:10:011
1446399162,10cyclictest5989-21latency08:05:231
1446399162,10cyclictest5989-21latency08:05:231
1446399162,10cyclictest27809-21latency_hist09:05:011
1446399161,9cyclictest30045-21basename08:00:011
1446399161,12cyclictest27216-21irqrtprio08:30:181
1446399161,12cyclictest27216-21irqrtprio08:30:171
1446399161,11cyclictest9177-21irqrtprio08:45:181
1446399161,11cyclictest13467-21acpi08:50:111
14455991610,4cyclictest6952-21tail11:20:170
14455991610,4cyclictest0-21swapper/007:25:230
1445599160,13cyclictest21440-21kthreadcore07:15:200
1448299151,8cyclictest0-21swapper/307:25:283
1445599151,11cyclictest31491-21apt-get08:35:120
1445599151,11cyclictest14460-21latency_hist07:10:010
1448299148,4cyclictest28366irq/58-eth008:05:143
1448299148,4cyclictest28366irq/58-eth008:05:143
1448299141,7cyclictest0-21swapper/309:00:013
1448299138,3cyclictest0-21swapper/307:35:113
1448299138,3cyclictest0-21swapper/307:35:113
1448299131,10cyclictest0-21swapper/307:35:013
1448299130,9cyclictest0-21swapper/308:10:133
1448299111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional