You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-19 - 23:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Wed Aug 19, 2026 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14964994635,7cyclictest321rcuc/309:20:143
14964994635,7cyclictest321rcuc/309:20:143
1496499417,21cyclictest35966irq/58-eth007:20:143
1496499382,24cyclictest35966irq/58-eth012:10:133
1495699380,25cyclictest6762-21ssh11:00:132
1496499371,21cyclictest35966irq/58-eth009:25:143
1496499361,18cyclictest35966irq/58-eth009:40:133
14964993419,9cyclictest35966irq/58-eth009:30:133
1495699340,18cyclictest22573-21ssh12:20:162
1496499336,18cyclictest35966irq/58-eth012:15:133
1496499331,21cyclictest35966irq/58-eth009:05:143
1496499331,20cyclictest35966irq/58-eth009:45:163
1495699330,23cyclictest9255-21ssh11:55:142
1496499322,21cyclictest35966irq/58-eth010:00:143
14956993211,15cyclictest27-21ksoftirqd/211:53:252
14948993221,6cyclictest4023-21apt-get10:05:141
1493699321,21cyclictest18421-21ssh12:00:140
1493699321,20cyclictest30018-21ssh10:55:160
1496499311,17cyclictest35966irq/58-eth011:40:153
1495699316,17cyclictest27-21ksoftirqd/211:40:252
1495699316,15cyclictest27-21ksoftirqd/210:00:242
1495699315,18cyclictest27-21ksoftirqd/212:32:522
14956993111,14cyclictest0-21swapper/210:35:562
14948993120,8cyclictest19959-21apt-get07:15:141
14948993117,10cyclictest22484-21apt-get08:25:131
14936993113,7cyclictest12-21ksoftirqd/011:16:210
1495699307,17cyclictest27-21ksoftirqd/209:44:042
1495699307,11cyclictest27-21ksoftirqd/212:03:302
1495699305,14cyclictest27-21ksoftirqd/211:10:212
1495699302,13cyclictest27-21ksoftirqd/212:35:272
14956993011,5cyclictest27-21ksoftirqd/212:07:122
1496499292,18cyclictest35966irq/58-eth011:15:143
1495699299,8cyclictest27-21ksoftirqd/211:08:272
1495699294,15cyclictest27-21ksoftirqd/209:10:112
14956992912,7cyclictest27-21ksoftirqd/210:34:022
1495699291,22cyclictest20925-21ssh10:50:122
14956992912,12cyclictest27-21ksoftirqd/212:25:472
14956992912,11cyclictest27-21ksoftirqd/210:11:212
14956992910,13cyclictest0-21swapper/210:58:072
14956992910,13cyclictest0-21swapper/209:35:552
14948992924,2cyclictest27279-21apt-get08:30:131
14948992921,4cyclictest20910-21apt-get10:50:121
14948992919,4cyclictest26047-21apt-get10:35:131
14948992917,9cyclictest32114-21apt-get08:35:121
1493699299,7cyclictest12-21ksoftirqd/009:14:140
14936992918,5cyclictest28652-21apt-get09:05:130
14936992910,8cyclictest12-21ksoftirqd/010:04:310
1495699286,2cyclictest131rcu_preempt10:45:562
1495699286,14cyclictest27-21ksoftirqd/211:49:102
1495699285,17cyclictest27-21ksoftirqd/211:15:102
1495699284,11cyclictest27-21ksoftirqd/212:10:212
1495699284,11cyclictest27-21ksoftirqd/210:19:052
1495699283,4cyclictest27-21ksoftirqd/209:58:052
14956992813,7cyclictest7332-21ssh10:24:512
1494899285,9cyclictest772-21diskmemload10:47:251
1494899285,8cyclictest772-21diskmemload09:22:121
1494899285,8cyclictest772-21diskmemload09:22:111
14948992815,10cyclictest19042-21apt-get08:55:131
1493699288,3cyclictest131rcu_preempt12:13:560
14936992812,9cyclictest0-21swapper/010:34:390
14936992812,6cyclictest12-21ksoftirqd/010:13:340
14936992812,10cyclictest0-21swapper/010:08:380
1496499278,11cyclictest35966irq/58-eth012:00:153
1496499273,17cyclictest0-21swapper/311:53:163
1496499272,14cyclictest35966irq/58-eth010:05:143
1495699278,12cyclictest27-21ksoftirqd/211:23:002
1495699276,14cyclictest27-21ksoftirqd/209:33:392
1495699274,12cyclictest131rcu_preempt08:00:242
1495699272,8cyclictest27-21ksoftirqd/210:41:192
14956992711,4cyclictest27-21ksoftirqd/212:17:452
14956992711,10cyclictest0-21swapper/211:31:022
14956992710,9cyclictest0-21swapper/211:25:132
14956992710,11cyclictest0-21swapper/210:05:412
14956992710,11cyclictest0-21swapper/209:22:132
14956992710,11cyclictest0-21swapper/209:22:132
1494899274,7cyclictest10643-21ssh11:20:121
1494899274,14cyclictest0-21swapper/111:50:121
14948992719,4cyclictest24925-21sh11:10:151
14948992718,7cyclictest9247-21apt-get11:55:141
14948992716,8cyclictest22433-21ssh12:20:131
14948992716,7cyclictest18423-21apt-get12:00:141
1493699274,4cyclictest12-21ksoftirqd/010:50:460
1493699274,4cyclictest12-21ksoftirqd/008:40:250
1493699273,5cyclictest12-21ksoftirqd/012:09:590
1493699273,5cyclictest12-21ksoftirqd/011:55:140
1493699273,4cyclictest12-21ksoftirqd/008:35:180
1493699273,10cyclictest12-21ksoftirqd/009:36:060
14936992712,9cyclictest0-21swapper/009:57:570
14936992711,7cyclictest0-21swapper/011:21:500
14936992710,9cyclictest12-21ksoftirqd/012:36:230
14936992710,9cyclictest12-21ksoftirqd/009:20:380
14936992710,9cyclictest12-21ksoftirqd/009:20:370
14936992710,3cyclictest131rcu_preempt11:04:330
1495699267,8cyclictest27-21ksoftirqd/209:16:222
1495699265,9cyclictest27-21ksoftirqd/209:25:172
1495699265,6cyclictest27-21ksoftirqd/210:29:152
1495699265,13cyclictest16623-21kworker/2:009:54:342
1495699265,12cyclictest15820-21kthreadcore07:10:192
1495699265,12cyclictest15820-21kthreadcore07:10:192
1495699262,4cyclictest27-21ksoftirqd/207:35:272
1494899269,14cyclictest0-21swapper/109:35:131
1494899266,8cyclictest8412-21ssh09:49:361
1494899264,8cyclictest14862-21irqstats08:50:181
14948992620,3cyclictest1549-21apt-get11:15:131
14948992618,3cyclictest19689-21apt-get11:25:121
14948992613,8cyclictest21-21ksoftirqd/107:10:141
14948992613,8cyclictest21-21ksoftirqd/107:10:131
14948992613,7cyclictest21-21ksoftirqd/110:00:131
1493699269,7cyclictest12-21ksoftirqd/008:10:170
1493699267,3cyclictest131rcu_preempt11:43:220
1493699266,3cyclictest131rcu_preempt12:20:140
1493699265,3cyclictest131rcu_preempt10:28:170
1493699264,5cyclictest12-21ksoftirqd/010:47:470
1493699264,5cyclictest12-21ksoftirqd/009:25:170
1493699264,4cyclictest12-21ksoftirqd/010:35:390
1493699263,5cyclictest12-21ksoftirqd/011:07:010
1493699263,5cyclictest12-21ksoftirqd/008:05:170
1493699263,4cyclictest12-21ksoftirqd/011:35:170
1493699263,4cyclictest12-21ksoftirqd/011:14:040
1493699263,4cyclictest12-21ksoftirqd/009:15:170
14936992612,8cyclictest0-21swapper/010:24:420
14936992611,4cyclictest12-21ksoftirqd/010:19:430
1493699261,10cyclictest0-21swapper/008:00:140
14936992610,9cyclictest0-21swapper/012:33:060
1496499257,12cyclictest0-21swapper/312:23:173
1496499254,12cyclictest35966irq/58-eth010:40:133
1496499251,17cyclictest35966irq/58-eth010:15:153
1496499251,14cyclictest35966irq/58-eth010:30:123
1495699258,11cyclictest27-21ksoftirqd/211:35:302
1495699258,11cyclictest27-21ksoftirqd/208:06:482
1495699257,11cyclictest27-21ksoftirqd/207:45:182
1495699256,12cyclictest27-21ksoftirqd/209:45:142
1495699256,10cyclictest27-21ksoftirqd/207:25:012
1495699255,9cyclictest27-21ksoftirqd/208:40:162
1495699253,12cyclictest131rcu_preempt08:30:302
1495699252,9cyclictest27-21ksoftirqd/209:05:292
1495699252,6cyclictest27-21ksoftirqd/208:10:152
1495699250,10cyclictest27-21ksoftirqd/208:15:242
1494899259,12cyclictest0-21swapper/108:20:121
1494899256,8cyclictest12206-21rm11:37:541
1494899256,7cyclictest12398-21ssh11:01:261
1494899256,14cyclictest0-21swapper/110:40:141
1494899254,8cyclictest13445-21ssh12:30:561
1494899253,9cyclictest17497-21/usr/sbin/munin12:15:301
1494899253,8cyclictest32040-21ntp_192.168.11409:05:261
14948992521,2cyclictest23511-21apt-get11:45:111
14948992520,3cyclictest7873-21apt-get10:25:121
14948992520,2cyclictest31254-21apt-get10:20:141
14948992518,3cyclictest8054-21apt-get08:10:121
14948992510,13cyclictest0-21swapper/107:45:121
1493699259,5cyclictest12-21ksoftirqd/009:30:300
1493699255,4cyclictest12-21ksoftirqd/007:35:120
1493699255,3cyclictest131rcu_preempt09:53:000
1493699255,2cyclictest131rcu_preempt11:26:520
1493699254,4cyclictest12-21ksoftirqd/012:15:170
1493699254,3cyclictest131rcu_preempt11:33:440
1493699253,9cyclictest12-21ksoftirqd/009:49:270
1493699253,4cyclictest12-21ksoftirqd/012:27:210
1493699253,4cyclictest12-21ksoftirqd/011:50:160
1493699253,4cyclictest12-21ksoftirqd/010:40:480
1493699253,4cyclictest12-21ksoftirqd/008:20:150
1493699253,4cyclictest12-21ksoftirqd/007:10:290
1493699253,4cyclictest12-21ksoftirqd/007:10:280
1493699252,5cyclictest12-21ksoftirqd/011:45:220
1493699252,4cyclictest12-21ksoftirqd/008:25:300
1493699252,4cyclictest12-21ksoftirqd/008:15:260
14936992518,3cyclictest9441-21apt-get08:45:130
14936992518,3cyclictest9441-21apt-get08:45:130
1496499243,10cyclictest35966irq/58-eth010:20:123
14964992417,5cyclictest0-21swapper/312:30:153
1496499241,15cyclictest0-21swapper/312:08:543
1496499241,11cyclictest35966irq/58-eth009:15:133
1495699249,4cyclictest27-21ksoftirqd/207:55:032
14956992411,10cyclictest0-21swapper/208:45:162
14956992411,10cyclictest0-21swapper/208:45:152
1494899249,12cyclictest29742-21tr09:25:201
1494899246,13cyclictest0-21swapper/108:05:131
1494899245,6cyclictest772-21diskmemload12:11:581
1494899244,15cyclictest9148-21ssh09:50:101
1494899242,8cyclictest0-21swapper/111:42:441
1494899242,7cyclictest0-21swapper/111:30:091
14948992418,3cyclictest10331-21apt-get09:15:141
14948992414,8cyclictest23452-21ssh09:40:131
14948992410,11cyclictest30602-21/usr/sbin/munin07:55:301
14948992410,11cyclictest0-21swapper/109:10:131
1493699247,5cyclictest12-21ksoftirqd/009:40:140
1493699247,3cyclictest131rcu_preempt09:00:190
1493699245,3cyclictest131rcu_preempt08:50:170
1493699245,3cyclictest131rcu_preempt07:45:150
1493699244,4cyclictest12-21ksoftirqd/007:55:160
1493699244,4cyclictest12-21ksoftirqd/007:50:170
1493699244,4cyclictest12-21ksoftirqd/007:40:270
1493699243,4cyclictest12-21ksoftirqd/007:30:270
1493699243,4cyclictest12-21ksoftirqd/007:20:240
1493699243,4cyclictest12-21ksoftirqd/007:15:210
1493699242,4cyclictest12-21ksoftirqd/008:55:170
1496499232,11cyclictest35966irq/58-eth010:55:153
1496499231,14cyclictest35966irq/58-eth011:00:133
1496499231,12cyclictest35966irq/58-eth011:55:133
1496499231,12cyclictest35966irq/58-eth009:10:133
1495699235,12cyclictest27-21ksoftirqd/207:33:072
1495699234,3cyclictest131rcu_preempt07:40:282
1495699233,8cyclictest27-21ksoftirqd/208:25:142
1495699232,4cyclictest27-21ksoftirqd/207:15:232
1495699232,12cyclictest131rcu_preempt09:00:232
14956992312,8cyclictest0-21swapper/208:20:132
1494899239,10cyclictest18343-21diskstats09:55:171
1494899237,13cyclictest30525-21kthreadcore12:05:211
1494899235,6cyclictest772-21diskmemload12:25:241
1494899234,4cyclictest5066-21ssh09:30:021
1494899233,7cyclictest772-21diskmemload10:18:161
1494899232,6cyclictest0-21swapper/112:38:551
1494899231,7cyclictest16991-21apt-get10:30:121
14948992311,8cyclictest29569-21apt-get07:25:141
1493699233,4cyclictest12-21ksoftirqd/008:30:130
1496499224,11cyclictest35966irq/58-eth011:45:103
1496499223,13cyclictest0-21swapper/312:38:163
1496499221,12cyclictest35966irq/58-eth012:25:123
1495699223,5cyclictest27-21ksoftirqd/207:50:202
1495699222,5cyclictest27-21ksoftirqd/208:55:252
1494899224,4cyclictest25400-21irqstats07:20:191
1494899224,3cyclictest0-21swapper/110:55:021
1494899223,4cyclictest21-21ksoftirqd/110:11:431
1494899223,14cyclictest1848-21ntp_192.168.11408:00:241
1494899222,4cyclictest0-21swapper/108:15:251
1494899221,7cyclictest15719-21ssh11:05:061
1494899221,7cyclictest0-21swapper/107:50:211
1494899221,5cyclictest0-21swapper/107:30:121
1493699224,4cyclictest12-21ksoftirqd/007:25:260
14964992111,6cyclictest35966irq/58-eth011:35:143
1495699215,8cyclictest27-21ksoftirqd/208:35:252
1495699214,7cyclictest27-21ksoftirqd/208:50:242
1495699211,11cyclictest27-21ksoftirqd/207:26:472
1494899219,9cyclictest9718-21if_eth008:45:181
1494899219,9cyclictest9718-21if_eth008:45:171
1494899212,10cyclictest8603-21kthreadcore07:35:221
1494899211,5cyclictest0-21swapper/107:40:121
1496499208,9cyclictest0-21swapper/310:10:223
14964992014,4cyclictest0-21swapper/310:35:153
14948992010,7cyclictest24482-21irqstats09:00:181
14948992010,6cyclictest4724-21diskstats08:40:151
1496499199,8cyclictest0-21swapper/311:28:553
1496499198,6cyclictest0-21swapper/311:30:143
1496499194,10cyclictest35966irq/58-eth010:45:153
14964991913,4cyclictest0-21swapper/311:20:153
14964991913,4cyclictest0-21swapper/311:10:133
14964991913,4cyclictest0-21swapper/311:05:153
14964991913,4cyclictest0-21swapper/310:55:003
14964991912,5cyclictest0-21swapper/310:28:123
1496499191,12cyclictest0-21swapper/308:35:223
1496499191,11cyclictest35966irq/58-eth009:35:143
1496499182,10cyclictest0-21swapper/308:55:143
1496499182,10cyclictest0-21swapper/308:30:133
1496499182,10cyclictest0-21swapper/308:10:133
14964991812,4cyclictest0-21swapper/309:55:313
14964991812,4cyclictest0-21swapper/308:45:283
14964991812,4cyclictest0-21swapper/308:45:273
1496499181,12cyclictest0-21swapper/307:30:163
1496499181,11cyclictest35966irq/58-eth007:15:113
14964991810,6cyclictest0-21swapper/309:52:593
14964991712,3cyclictest0-21swapper/307:45:243
1496499171,13cyclictest0-21swapper/307:35:143
1496499171,11cyclictest0-21swapper/308:40:043
1495699171,13cyclictest15064-21latency_hist07:10:012
1496499161,10cyclictest0-21swapper/307:40:133
14964991610,4cyclictest0-21swapper/308:05:173
1496499159,4cyclictest0-21swapper/308:00:283
1496499151,11cyclictest0-21swapper/309:00:183
1496499151,10cyclictest0-21swapper/308:50:153
1496499151,10cyclictest0-21swapper/307:50:133
1496499149,3cyclictest0-21swapper/307:55:163
1496499148,4cyclictest0-21swapper/307:25:253
1496499141,9cyclictest0-21swapper/308:15:153
1494899142,9cyclictest14884-21latency_hist07:10:011
1496499137,4cyclictest33-21ksoftirqd/308:25:143
1496499137,4cyclictest0-21swapper/308:20:193
1496499136,5cyclictest33-21ksoftirqd/307:10:143
1496499136,5cyclictest33-21ksoftirqd/307:10:133
1493699131,10cyclictest14931-21cyclictest07:10:020
1496499115,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional