You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 23:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Wed Jul 29, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3394998035,33cyclictest35966irq/58-eth011:25:133
3369996422,12cyclictest12-21ksoftirqd/010:35:130
336999541,44cyclictest28373-21ssh11:20:150
336999518,6cyclictest131rcu_preempt07:15:140
3386994823,15cyclictest0-21swapper/211:25:152
336999483,3cyclictest131rcu_preempt11:25:130
336999483,38cyclictest30342-21ssh12:15:140
3369994815,5cyclictest12-21ksoftirqd/010:05:130
338699454,2cyclictest131rcu_preempt11:10:152
3386994413,19cyclictest27-21ksoftirqd/208:10:122
3386994413,19cyclictest27-21ksoftirqd/208:10:122
338699431,33cyclictest14207-21ssh10:00:152
339499403,26cyclictest35966irq/58-eth010:35:133
339499402,24cyclictest35966irq/58-eth010:25:143
338699392,27cyclictest19820-21ssh09:45:142
339499382,22cyclictest35966irq/58-eth012:10:143
339499382,22cyclictest35966irq/58-eth012:10:133
339499382,22cyclictest35966irq/58-eth011:30:143
3378993824,10cyclictest6056-21apt-get08:20:131
339499373,19cyclictest35966irq/58-eth011:20:153
3378993728,4cyclictest22960-21apt-get11:35:151
3394993522,7cyclictest35966irq/58-eth012:30:143
3378993519,11cyclictest0-21swapper/108:25:131
336999353,20cyclictest12-21ksoftirqd/007:55:150
338699345,10cyclictest22971-21ssh11:35:142
337899333,21cyclictest0-21swapper/108:45:121
337899333,21cyclictest0-21swapper/108:45:121
3378993326,3cyclictest28676-21apt-get09:50:151
3369993315,6cyclictest131rcu_preempt09:45:140
3394993217,9cyclictest35966irq/58-eth010:55:143
3378993215,14cyclictest0-21swapper/108:35:121
3394993112,13cyclictest35966irq/58-eth010:20:123
3386993115,9cyclictest27-21ksoftirqd/209:35:142
3378993122,5cyclictest9598-21apt-get07:50:141
337899312,23cyclictest16378-21ssh09:25:131
337899312,23cyclictest16378-21ssh09:25:131
3378993116,10cyclictest0-21swapper/112:05:111
336999312,20cyclictest8791-21ssh10:15:140
339499303,18cyclictest0-21swapper/311:45:133
338699303,21cyclictest24165-21ssh09:29:082
338699303,21cyclictest24165-21ssh09:29:072
338699302,21cyclictest17652-21ssh10:20:132
338699302,21cyclictest0-21swapper/212:10:162
338699302,21cyclictest0-21swapper/212:10:152
337899302,24cyclictest21995-21ssh09:10:161
337899302,15cyclictest26704-21ssh11:55:141
3378993020,4cyclictest13968-21apt-get11:30:141
336999301,22cyclictest22986-21ssh11:35:140
339499292,17cyclictest35966irq/58-eth010:50:133
3386992921,3cyclictest27-21ksoftirqd/207:50:132
337899299,15cyclictest0-21swapper/111:00:151
3378992923,3cyclictest10347-21apt-get11:10:151
337899292,22cyclictest20922-21ssh10:40:121
337899292,20cyclictest16000-21ssh12:25:151
3378992919,7cyclictest1224-21ssh11:05:111
336999292,21cyclictest31216-21ssh12:32:100
3394992816,9cyclictest35966irq/58-eth012:00:153
339499281,14cyclictest35966irq/58-eth007:10:123
338699283,20cyclictest0-21swapper/211:24:072
338699281,7cyclictest24947-1kworker/u9:212:30:142
337899282,11cyclictest18170-21sed10:20:171
3378992818,8cyclictest1554-21ssh12:35:111
339499272,14cyclictest35966irq/58-eth011:15:123
338699272,20cyclictest0-21swapper/210:29:082
338699272,19cyclictest0-21swapper/208:20:132
338699272,19cyclictest0-21swapper/207:45:152
337899272,15cyclictest3335-21ssh12:00:141
3378992720,4cyclictest8389-21apt-get07:15:131
3378992717,6cyclictest21367-21apt-get12:10:161
3378992717,6cyclictest21367-21apt-get12:10:151
3378992714,10cyclictest15937-21find10:35:291
336999272,19cyclictest26619-21ssh10:25:130
339499269,11cyclictest35966irq/58-eth009:45:143
339499263,15cyclictest0-21swapper/309:23:393
339499262,16cyclictest35966irq/58-eth010:45:123
338699262,20cyclictest3365-21cyclictest12:04:052
338699261,10cyclictest8476-21ssh10:31:092
337899262,19cyclictest29898-21ssh10:45:071
337899262,18cyclictest8635-21ssh11:45:151
337899262,10cyclictest24279-21latency_hist11:00:011
3378992617,7cyclictest30514-21find10:25:291
336999264,15cyclictest12-21ksoftirqd/011:30:140
336999261,18cyclictest28692-21ssh09:50:130
338699251,9cyclictest0-21swapper/212:35:142
338699251,17cyclictest10929-21ssh09:40:142
338699250,11cyclictest24704-21ssh11:00:142
337899252,17cyclictest25318-21sh09:30:151
3378992518,4cyclictest13189-21apt-get07:20:151
337899251,19cyclictest10882-21ssh10:34:081
339499247,14cyclictest35966irq/58-eth011:50:133
339499245,13cyclictest0-21swapper/312:20:123
339499243,15cyclictest0-21swapper/309:29:083
339499243,15cyclictest0-21swapper/309:29:073
339499242,15cyclictest35966irq/58-eth012:15:143
3394992411,8cyclictest35966irq/58-eth009:30:143
338699241,8cyclictest12009-21ssh10:35:132
338699241,17cyclictest27346irq/40-dwc2_hso09:20:132
338699241,13cyclictest30341-21ssh12:15:132
338699240,9cyclictest5284-21ssh10:48:532
337899249,9cyclictest0-21swapper/112:20:131
337899248,11cyclictest0-21swapper/110:00:141
337899247,12cyclictest15604-21ntp_kernel_err09:00:251
337899244,4cyclictest19793-21apt-get09:45:121
3378992418,3cyclictest4954-21apt-get11:25:141
3378992417,3cyclictest10918-21apt-get09:40:131
339499232,15cyclictest0-21swapper/311:00:153
3394992311,7cyclictest35966irq/58-eth009:35:143
338699232,6cyclictest0-21swapper/210:57:092
338699232,6cyclictest0-21swapper/210:53:532
338699232,16cyclictest0-21swapper/212:25:102
338699231,9cyclictest5569-21irqcore09:55:172
338699231,18cyclictest3365-21cyclictest12:07:092
338699231,15cyclictest0-21swapper/209:30:192
338699231,14cyclictest27346irq/40-dwc2_hso08:15:112
3386992310,11cyclictest614-21dbus-daemon10:18:472
338699230,15cyclictest26378-21ls11:55:012
337899236,8cyclictest0-21swapper/109:17:271
337899236,3cyclictest0-21swapper/108:05:241
337899232,16cyclictest25007-21ssh12:30:151
3378992318,2cyclictest7412-21apt-get09:20:151
3378992317,3cyclictest28829-21apt-get08:10:141
3378992317,3cyclictest28829-21apt-get08:10:131
337899231,17cyclictest0-21swapper/109:35:141
337899231,16cyclictest32229-21sh10:10:141
3369992314,6cyclictest10350-21ssh11:10:140
336999231,17cyclictest3119-21find10:30:120
336999230,19cyclictest12541-21ls11:45:300
339499223,13cyclictest0-21swapper/310:34:083
3394992211,8cyclictest0-21swapper/310:08:453
338699229,10cyclictest500-21diskmemload09:15:212
338699222,18cyclictest0-21swapper/209:50:152
338699222,14cyclictest0-21swapper/207:55:152
338699221,16cyclictest0-21swapper/211:40:082
338699221,16cyclictest0-21swapper/210:42:082
338699221,11cyclictest21560-21diskmemload11:55:272
337899228,4cyclictest0-21swapper/107:25:231
337899228,10cyclictest3912-21irqcore07:10:181
337899228,10cyclictest31402-21unixbench_multi07:35:291
337899225,8cyclictest0-21swapper/111:52:561
337899223,5cyclictest21-21ksoftirqd/111:19:031
337899223,14cyclictest1217-21apt-get08:15:111
3378992212,6cyclictest30324-21apt-get12:15:131
3378992210,9cyclictest0-21swapper/110:15:131
336999221,18cyclictest30804-21find09:15:110
336999221,17cyclictest6056-21apt-get08:20:110
336999221,17cyclictest11457-21ls09:20:300
336999221,16cyclictest22939-21ls10:05:010
339499213,15cyclictest0-21swapper/310:00:143
339499213,11cyclictest0-21swapper/311:10:163
339499212,17cyclictest0-21swapper/310:43:073
339499211,14cyclictest0-21swapper/311:05:113
338699212,9cyclictest20316-21kthreadcore07:25:232
338699212,15cyclictest0-21swapper/211:07:092
338699211,6cyclictest21560-21diskmemload09:13:432
3386992113,5cyclictest21231-21ssh11:15:222
338699211,15cyclictest0-21swapper/211:30:132
338699211,11cyclictest26450-21kthreadcore10:05:232
338699210,9cyclictest22973-21timerwakeupswit08:00:282
338699210,12cyclictest131rcu_preempt08:50:012
338699210,12cyclictest131rcu_preempt08:50:002
338699210,11cyclictest9796-21cut12:20:222
337899217,10cyclictest16992-21dpkg09:05:121
337899215,4cyclictest21-21ksoftirqd/111:21:221
337899210,8cyclictest4786-21ssh11:40:541
336999212,16cyclictest21380-21ssh11:50:270
3369992114,4cyclictest0-21swapper/007:10:220
3369992112,6cyclictest15718-21ssh10:55:150
3369992111,7cyclictest6031-21cat09:35:300
336999211,17cyclictest12292-21find12:05:110
336999211,16cyclictest16990-21apt-get09:05:130
336999211,16cyclictest16533-21cut09:25:160
336999211,16cyclictest16533-21cut09:25:150
339499203,11cyclictest0-21swapper/312:25:133
3394992011,6cyclictest35966irq/58-eth010:15:123
3394992010,6cyclictest0-21swapper/310:10:023
338699208,5cyclictest26821-21cut08:40:212
338699201,9cyclictest11985-21cat07:15:292
338699201,3cyclictest131rcu_preempt08:05:172
338699201,10cyclictest8157-21cron07:15:012
338699201,10cyclictest18996-21cat08:30:232
337899208,9cyclictest6446-21unixbench_singl08:50:301
337899208,9cyclictest23365-21fschecks_count10:05:171
337899207,9cyclictest6699-21cat10:50:151
337899206,3cyclictest0-21swapper/108:00:131
3378992016,2cyclictest15675-21apt-get08:30:151
3378992012,5cyclictest4558-21munin-plugin-st07:45:011
3378992011,6cyclictest0-21swapper/109:57:261
336999203,12cyclictest0-21swapper/009:00:130
3369992014,3cyclictest0-21swapper/011:15:130
3369992011,6cyclictest8227-21ssh11:45:020
3369992011,6cyclictest6606-21ssh10:50:130
336999201,16cyclictest32223-21rm10:10:130
339499197,9cyclictest0-21swapper/311:57:213
339499194,11cyclictest35966irq/58-eth012:05:133
339499193,7cyclictest35966irq/58-eth011:35:143
339499192,11cyclictest0-21swapper/312:35:103
339499192,11cyclictest0-21swapper/311:40:093
339499192,11cyclictest0-21swapper/309:10:153
339499192,10cyclictest0-21swapper/308:10:143
339499192,10cyclictest0-21swapper/308:10:133
339499191,12cyclictest0-21swapper/309:00:283
339499191,11cyclictest0-21swapper/307:50:243
338699195,3cyclictest0-21swapper/209:05:172
338699191,9cyclictest4612-21latency_hist07:45:012
338699191,8cyclictest27782-21fschecks_time07:35:172
338699191,15cyclictest12364-21fschecks_time09:00:162
338699191,15cyclictest11859-21kthreadcore08:25:202
338699191,15cyclictest10891-21wc08:55:262
338699191,14cyclictest0-21swapper/207:20:142
338699191,10cyclictest8746-21df_inode11:45:162
338699191,10cyclictest1662-21kthreadcore10:10:212
337899197,9cyclictest19210-21latency_hist08:00:031
337899196,9cyclictest22761-21/usr/sbin/munin07:30:191
337899195,7cyclictest21-21ksoftirqd/108:55:131
337899191,4cyclictest53646usb-storage08:40:111
336999199,7cyclictest7350-21ssh12:20:170
336999199,7cyclictest19327-21diskstats08:00:160
336999199,7cyclictest1-21systemd10:48:540
336999197,9cyclictest27563-21kthreadcore11:00:230
336999197,3cyclictest21560-21diskmemload10:23:310
336999196,8cyclictest0-21swapper/007:25:150
336999191,6cyclictest21560-21diskmemload12:00:490
3369991911,6cyclictest4973-21ssh11:05:260
3369991911,5cyclictest3483-21ssh12:35:210
336999191,15cyclictest25678-21ssh09:10:280
336999190,8cyclictest5252-21cpuspeed09:55:140
339499182,10cyclictest36066irq/59-eth009:40:133
339499182,10cyclictest0-21swapper/308:40:053
339499182,10cyclictest0-21swapper/308:30:143
339499182,10cyclictest0-21swapper/307:55:153
339499182,10cyclictest0-21swapper/307:45:143
338699181,14cyclictest25005-21idleruntime-cro08:40:012
338699181,13cyclictest22779-21apt-get07:30:152
338699181,10cyclictest2586-21apt-get08:50:122
337899189,6cyclictest4788-21apt-get07:45:131
336999189,6cyclictest23616-21kthreadcore07:30:210
336999186,9cyclictest27865-21grep08:05:290
336999183,10cyclictest0-21swapper/012:26:100
336999181,9cyclictest30040-21apt-key08:45:010
336999181,7cyclictest19561-21cat09:45:020
3369991812,3cyclictest25325-21cpu09:30:140
336999181,15cyclictest6899-21kthreadcore07:45:200
336999181,13cyclictest20923-21apt-get10:40:110
3369991810,6cyclictest2886-21cron12:00:010
3369991810,5cyclictest32577-21wc08:10:280
3369991810,5cyclictest32577-21wc08:10:270
339499177,8cyclictest0-21swapper/309:55:403
339499172,10cyclictest0-21swapper/308:45:053
339499172,10cyclictest0-21swapper/308:45:043
339499172,10cyclictest0-21swapper/308:00:153
339499171,8cyclictest0-21swapper/307:20:123
339499171,13cyclictest0-21swapper/308:30:023
338699171,8cyclictest3443-21latency_hist07:10:022
336999178,7cyclictest1188-21ntp_192.168.11408:45:240
336999178,7cyclictest1188-21ntp_192.168.11408:45:230
336999176,9cyclictest30366-21cut07:35:210
336999173,11cyclictest17120-21kthreadcore08:30:200
336999172,12cyclictest2466-21kthreadcore07:40:230
336999171,7cyclictest21344-21/usr/sbin/munin12:10:310
336999171,7cyclictest21344-21/usr/sbin/munin12:10:310
336999171,2cyclictest131rcu_preempt08:15:120
3369991712,3cyclictest12-21ksoftirqd/008:35:130
3369991710,5cyclictest13092-21pluginstate07:50:280
339499168,6cyclictest0-21swapper/309:15:513
339499161,12cyclictest0-21swapper/307:40:243
3394991610,4cyclictest0-21swapper/309:50:083
336999163,10cyclictest3505-21latency_hist07:10:020
336999162,9cyclictest0-21swapper/007:20:140
336999161,12cyclictest9756-21tr08:55:200
336999161,11cyclictest10865-21apt-get08:25:150
339499151,7cyclictest0-21swapper/307:20:013
339499151,12cyclictest0-21swapper/308:10:013
336999158,5cyclictest2581-21acpi08:50:120
339499142,10cyclictest0-21swapper/307:30:163
339499141,7cyclictest0-21swapper/308:15:143
339499141,7cyclictest0-21swapper/307:35:163
339499141,10cyclictest0-21swapper/308:40:013
339499141,10cyclictest0-21swapper/308:20:223
337899141,10cyclictest3417-21latency_hist07:10:011
339499135,5cyclictest0-21swapper/308:50:153
339499132,7cyclictest0-21swapper/307:25:143
339499131,8cyclictest35966irq/58-eth009:05:143
339499121,4cyclictest0-21swapper/308:55:183
339499111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional