You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 14:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Jul 28, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2976399552,42cyclictest5387-21ssh11:10:141
29779994741,4cyclictest35966irq/58-eth012:10:143
2975499471,41cyclictest30644-21apt-get12:00:130
2975499464,8cyclictest131rcu_preempt07:30:120
2975499461,42cyclictest0-21swapper/008:05:130
2975499460,43cyclictest15894-21ssh09:10:140
2975499451,41cyclictest8826-21apt-get10:00:140
2975499442,37cyclictest0-21swapper/008:55:130
2975499441,39cyclictest0-21swapper/011:05:150
29779994335,5cyclictest35966irq/58-eth009:30:123
2975499431,38cyclictest0-21swapper/012:10:120
2975499421,37cyclictest0-21swapper/010:40:130
2975499420,38cyclictest1727-21ssh10:50:150
2975499411,36cyclictest0-21swapper/010:55:120
2975499411,25cyclictest1677-21ssh11:43:110
2975499402,32cyclictest9006-21apt-get11:30:130
2975499402,32cyclictest9006-21apt-get11:30:130
2975499402,32cyclictest19722-21apt-get11:00:120
2975499401,36cyclictest23428-21ssh11:20:120
2975499401,35cyclictest0-21swapper/012:25:120
2975499401,32cyclictest0-21swapper/010:05:150
2977999396,20cyclictest35966irq/58-eth011:50:123
2975499392,30cyclictest0-21swapper/009:30:120
2975499391,24cyclictest9851-21ssh11:47:120
2975499391,24cyclictest11876-21acpi07:25:120
2975499391,21cyclictest21747-21ssh11:18:310
2975499390,36cyclictest26783-21ssh10:10:150
2975499390,36cyclictest26783-21ssh10:10:150
29771993815,17cyclictest0-21swapper/210:05:142
2975499381,33cyclictest0-21swapper/010:45:130
29779993714,19cyclictest36066irq/59-eth011:55:133
29771993727,7cyclictest19747-21apt-get12:30:132
29771993727,7cyclictest19747-21apt-get12:30:122
2976399371,29cyclictest8847-21ssh10:00:141
2975499371,33cyclictest32436-21apt-get11:25:140
2975499371,31cyclictest28848-21ssh12:35:150
2975499371,30cyclictest21442-21ssh10:25:130
2975499371,27cyclictest8132-21users07:50:310
2975499371,21cyclictest28316-21ssh11:57:420
2975499371,20cyclictest0-21swapper/011:50:040
2975499370,34cyclictest28699-21apt-get09:35:160
2975499370,24cyclictest27369-21sshd08:13:450
2976399362,19cyclictest12106-21ssh10:35:461
2975499361,32cyclictest0-21swapper/012:15:130
2975499361,31cyclictest14433-21apt-get08:35:110
2975499361,23cyclictest25055-21open_files07:35:270
2975499361,22cyclictest13229-21ssh10:37:070
2975499361,21cyclictest6848-21ssh09:21:070
2975499361,20cyclictest15496-21diskmemload10:22:310
2975499361,19cyclictest23784-21ssh11:36:340
2977199356,17cyclictest27554-21ssh10:27:022
29771993515,18cyclictest0-21swapper/209:40:122
29771993513,18cyclictest0-21swapper/208:30:122
29771993513,18cyclictest0-21swapper/208:30:112
2977199351,18cyclictest18636-21ssh09:29:252
2975499351,30cyclictest19746-21ssh12:30:130
2975499351,30cyclictest19746-21ssh12:30:130
2975499351,20cyclictest12681-21ssh11:13:320
2975499350,30cyclictest7148-21apt-get12:05:130
2975499350,24cyclictest30169-21ssh09:53:070
2977999343,27cyclictest35966irq/58-eth010:50:143
2977199345,15cyclictest6284-21ssh11:27:122
29771993413,13cyclictest0-21swapper/211:05:152
29771993411,19cyclictest0-21swapper/210:55:122
2975499342,29cyclictest14214-21apt-get09:45:120
2975499341,30cyclictest0-21swapper/008:45:130
2975499341,21cyclictest682-21ssh09:19:340
2975499341,18cyclictest0-21swapper/010:34:210
2975499340,30cyclictest5254-21apt-get09:40:120
2975499340,29cyclictest10541-21ssh09:25:140
2977999333,20cyclictest35966irq/58-eth010:55:133
29771993324,6cyclictest7043-21apt-get10:35:142
29771993320,9cyclictest0-21swapper/209:30:122
29771993320,11cyclictest28826-21apt-get12:35:152
29771993315,15cyclictest0-21swapper/210:00:132
29771993313,16cyclictest0-21swapper/210:50:142
2975499331,28cyclictest32150-21ssh09:55:120
2975499331,20cyclictest8530-21ssh10:15:540
2977999326,19cyclictest0-21swapper/311:15:123
2977199323,17cyclictest0-21swapper/211:58:222
29771993225,3cyclictest25065-21apt-get10:45:132
29771993221,8cyclictest27346irq/40-dwc2_hso12:00:142
29771993216,11cyclictest0-21swapper/208:05:122
29771993215,9cyclictest15787-21cron12:10:012
2975499323,26cyclictest2438-21fschecks_time07:15:160
2975499320,21cyclictest930-21snmpd07:21:450
2975499320,20cyclictest8581-21ssh12:22:530
2977199317,11cyclictest5414-21ssh10:33:342
29771993116,11cyclictest27346irq/40-dwc2_hso08:45:132
2975499311,26cyclictest11793-21netstat07:55:240
2977999303,17cyclictest35966irq/58-eth010:05:153
2977199304,18cyclictest15496-21diskmemload11:41:062
2977199303,11cyclictest22304-21ssh09:12:232
29771993014,5cyclictest1325-21apt-get08:55:132
29771993013,15cyclictest27346irq/40-dwc2_hso08:25:132
29771993010,15cyclictest0-21swapper/211:30:132
29771993010,15cyclictest0-21swapper/211:30:122
2975499301,26cyclictest9583-21ntp_states09:00:260
2975499301,26cyclictest3440-21ntp_192.168.11408:20:220
2975499301,26cyclictest2118-21memory07:45:220
2975499301,25cyclictest0-21swapper/008:50:120
2975499301,25cyclictest0-21swapper/007:40:120
2975499301,25cyclictest0-21swapper/007:40:110
2975499301,24cyclictest0-21swapper/008:15:140
29779992923,3cyclictest35966irq/58-eth012:35:143
2977999292,22cyclictest35966irq/58-eth010:00:133
2977999292,11cyclictest0-21swapper/310:25:133
29779992919,8cyclictest35966irq/58-eth011:00:133
2977999291,8cyclictest0-21swapper/309:10:133
29779992913,10cyclictest35966irq/58-eth012:05:003
2977199294,13cyclictest0-21swapper/211:14:212
29771992921,6cyclictest24879-21apt-get09:15:132
2977199291,6cyclictest28891-21ssh09:51:332
29771992914,13cyclictest27075-21kthreadcore07:40:202
29771992914,13cyclictest27075-21kthreadcore07:40:202
2976399292,20cyclictest26754-21ssh10:10:131
2976399292,20cyclictest26754-21ssh10:10:131
2976399291,19cyclictest14239-21ssh09:45:131
2975499291,25cyclictest0-21swapper/007:10:110
2977999281,19cyclictest35966irq/58-eth007:25:133
2977199287,5cyclictest0-21swapper/211:04:462
2977199287,10cyclictest15496-21diskmemload10:11:022
2977199287,10cyclictest15496-21diskmemload10:11:012
2977199286,9cyclictest28676-21/usr/sbin/munin09:35:182
2977199283,11cyclictest0-21swapper/209:59:522
2977199282,13cyclictest20316-21ssh09:46:572
2977199282,13cyclictest17632-21rm10:20:592
29771992813,12cyclictest26702-21usage_Anybus08:10:302
2977199280,23cyclictest6718-21kworker/u8:312:24:472
2976399282,21cyclictest18638-21sshd09:29:241
2975499281,24cyclictest13002-21needreboot08:30:230
2975499281,24cyclictest13002-21needreboot08:30:230
2977999273,8cyclictest0-21swapper/311:05:153
2977999271,4cyclictest0-21swapper/311:10:133
2977199277,8cyclictest0-21swapper/211:52:452
2977199273,9cyclictest23373-21ssh11:36:042
2977199272,13cyclictest3295-21ssh10:15:072
29771992713,11cyclictest0-21swapper/210:40:132
29771992712,8cyclictest30614-21cut07:10:182
29771992712,12cyclictest21751-21fschecks_time07:35:172
2976399273,3cyclictest131rcu_preempt09:35:131
2976399272,20cyclictest24899-21ssh09:15:121
2976399272,20cyclictest15801-21ssh12:25:461
29763992716,7cyclictest0-21swapper/107:35:141
2976399271,15cyclictest23427-21sh11:20:121
2975499274,20cyclictest11702-21kthreadcore09:05:210
2975499270,24cyclictest13428-21/usr/sbin/munin08:00:160
2975499270,23cyclictest19215-21/usr/sbin/munin08:40:130
2977999261,7cyclictest0-21swapper/310:35:263
2977999261,6cyclictest0-21swapper/312:30:303
2977999261,6cyclictest0-21swapper/312:30:303
2977999261,6cyclictest0-21swapper/309:47:043
2977999261,6cyclictest0-21swapper/309:20:363
2977999261,4cyclictest0-21swapper/310:15:013
2977999261,4cyclictest0-21swapper/310:15:003
2977199267,16cyclictest0-21swapper/208:35:122
2977199266,8cyclictest11908-21ssh11:49:412
2977199264,19cyclictest0-21swapper/211:21:372
2977199263,7cyclictest0-21swapper/212:25:102
29771992622,2cyclictest25131-21apt-get12:15:132
29771992613,9cyclictest0-21swapper/208:15:142
29771992612,11cyclictest8204-21latency_hist07:55:022
2976399266,10cyclictest21-21ksoftirqd/111:47:411
2976399263,6cyclictest21-21ksoftirqd/110:25:151
2976399262,18cyclictest10144-21sh12:24:471
2976399262,16cyclictest21670-21apt-get11:55:121
2976399261,19cyclictest6554-21ssh12:04:471
2975499261,22cyclictest9386-21sshd08:29:460
2977999255,5cyclictest0-21swapper/311:45:143
2977999254,14cyclictest35966irq/58-eth007:45:123
2977999252,9cyclictest0-21swapper/312:15:153
2977999252,9cyclictest0-21swapper/310:15:303
2977999252,4cyclictest0-21swapper/309:15:143
2977999252,13cyclictest0-21swapper/308:10:133
2977999251,7cyclictest0-21swapper/311:35:183
2977999251,6cyclictest0-21swapper/312:25:243
2977999251,6cyclictest0-21swapper/311:25:293
2977999251,6cyclictest0-21swapper/310:45:193
2977999251,6cyclictest0-21swapper/310:40:313
2977999251,10cyclictest0-21swapper/309:55:123
2977199259,8cyclictest2076-21latency07:45:212
2977199259,13cyclictest14993-21cat09:05:322
2977199258,9cyclictest23902-21date08:45:022
2977199258,7cyclictest10790-21latency_hist09:05:022
29771992510,12cyclictest18256-21cat11:15:282
2976399258,4cyclictest21-21ksoftirqd/112:36:491
2976399257,9cyclictest21-21ksoftirqd/112:06:491
2976399257,7cyclictest21-21ksoftirqd/111:51:091
2976399256,8cyclictest21-21ksoftirqd/112:34:201
2976399256,8cyclictest21-21ksoftirqd/112:34:191
2976399256,7cyclictest21-21ksoftirqd/110:05:511
2976399256,3cyclictest131rcu_preempt11:00:521
2976399255,5cyclictest21-21ksoftirqd/110:45:151
2976399255,3cyclictest131rcu_preempt11:30:161
2976399255,3cyclictest131rcu_preempt11:30:161
2976399255,3cyclictest131rcu_preempt07:50:261
2976399254,9cyclictest21-21ksoftirqd/111:27:071
2976399254,4cyclictest21-21ksoftirqd/110:55:141
2976399254,3cyclictest131rcu_preempt09:10:001
2976399253,5cyclictest21-21ksoftirqd/109:20:251
2975499252,20cyclictest29749-21cyclictest07:10:020
2977999246,3cyclictest0-21swapper/312:08:593
2977999245,4cyclictest35966irq/58-eth007:40:003
2977999245,12cyclictest0-21swapper/312:20:133
2977999243,6cyclictest0-21swapper/309:25:273
2977999242,9cyclictest0-21swapper/310:20:253
2977999241,8cyclictest0-21swapper/311:30:203
2977999241,8cyclictest0-21swapper/311:30:193
2977999241,7cyclictest0-21swapper/309:40:113
2977999241,7cyclictest0-21swapper/307:25:013
2977999241,6cyclictest0-21swapper/311:20:213
2977999241,5cyclictest0-21swapper/308:35:013
2977999241,5cyclictest0-21swapper/308:35:013
2977999241,5cyclictest0-21swapper/307:50:273
29779992414,6cyclictest0-21swapper/311:40:133
2977199248,8cyclictest11740-21/usr/sbin/munin07:55:242
2977199248,13cyclictest15496-21diskmemload12:10:082
29771992416,5cyclictest27346irq/40-dwc2_hso09:20:122
2976399249,10cyclictest201rcuc/110:24:441
2976399247,5cyclictest131rcu_preempt09:31:031
2976399247,4cyclictest21-21ksoftirqd/109:10:251
2976399244,8cyclictest1721-21ssh11:06:051
2976399244,4cyclictest21-21ksoftirqd/108:15:161
2976399243,4cyclictest21-21ksoftirqd/109:00:021
29763992413,6cyclictest201rcuc/109:40:131
2977999232,7cyclictest0-21swapper/307:40:163
2977999232,7cyclictest0-21swapper/307:40:153
2977999232,6cyclictest0-21swapper/308:25:213
2977999231,7cyclictest0-21swapper/309:35:163
2977999231,7cyclictest0-21swapper/307:55:273
2977999231,7cyclictest0-21swapper/307:10:203
2977999231,6cyclictest0-21swapper/309:50:233
2977999231,6cyclictest0-21swapper/309:10:013
2977999231,6cyclictest0-21swapper/308:15:193
2977999231,5cyclictest0-21swapper/308:40:173
2977999231,4cyclictest0-21swapper/310:30:173
2977199238,8cyclictest11465-21usage_EthcTask07:20:312
2977199237,12cyclictest0-21swapper/207:25:132
2977199236,3cyclictest0-21swapper/207:15:252
29771992317,3cyclictest28849-21apt-get08:50:132
2976399237,7cyclictest21-21ksoftirqd/111:43:131
2976399236,7cyclictest21-21ksoftirqd/111:18:371
2976399236,7cyclictest21-21ksoftirqd/110:18:561
2976399236,6cyclictest21-21ksoftirqd/109:51:031
2976399236,2cyclictest131rcu_preempt09:59:161
2976399233,4cyclictest21-21ksoftirqd/112:15:231
2976399233,3cyclictest0-21swapper/109:00:271
29763992316,4cyclictest201rcuc/111:35:131
29763992315,4cyclictest18010-21apt-get08:05:131
2976399231,4cyclictest131rcu_preempt10:30:051
29763992312,8cyclictest16187-21apt-get12:10:131
2976399230,4cyclictest21-21ksoftirqd/110:40:291
2977999222,15cyclictest0-21swapper/308:05:123
2977999221,7cyclictest0-21swapper/309:00:283
2977999221,7cyclictest0-21swapper/308:45:193
2977999221,7cyclictest0-21swapper/307:30:183
2977999221,6cyclictest0-21swapper/308:55:173
2977999221,6cyclictest0-21swapper/308:50:153
2977999221,6cyclictest0-21swapper/308:40:013
2977999221,6cyclictest0-21swapper/308:20:183
2977999221,6cyclictest0-21swapper/308:00:153
2977999221,6cyclictest0-21swapper/307:15:163
2977199227,12cyclictest3495-21ntp_offset08:20:232
2976399225,2cyclictest131rcu_preempt10:50:161
2976399224,7cyclictest19868-21irqstats08:40:191
2976399223,5cyclictest21-21ksoftirqd/107:25:161
2976399223,4cyclictest21-21ksoftirqd/108:34:141
2976399223,4cyclictest21-21ksoftirqd/108:34:131
29763992216,3cyclictest201rcuc/107:15:241
2977199211,6cyclictest0-21swapper/207:30:132
2976399216,4cyclictest131rcu_preempt07:55:261
2976399216,2cyclictest131rcu_preempt08:35:281
2976399215,3cyclictest131rcu_preempt08:10:171
2976399214,4cyclictest21-21ksoftirqd/108:20:281
2976399214,4cyclictest21-21ksoftirqd/107:20:181
2976399213,4cyclictest21-21ksoftirqd/108:00:271
2976399212,4cyclictest21-21ksoftirqd/107:45:231
29763992112,6cyclictest16721-21apt-get07:30:131
2977199209,8cyclictest17682-21cat08:00:302
2976399204,3cyclictest131rcu_preempt08:25:241
29763992011,6cyclictest0-21swapper/107:10:301
2976399195,3cyclictest131rcu_preempt08:45:171
2976399192,5cyclictest21-21ksoftirqd/108:50:241
2976399191,6cyclictest21-21ksoftirqd/107:40:151
2976399191,6cyclictest21-21ksoftirqd/107:40:151
2976399181,3cyclictest131rcu_preempt07:10:021
2977199151,10cyclictest29825-21latency_hist07:10:022
2977999131,9cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional