You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-10 - 03:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Aug 10, 2026 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12815996144,9cyclictest35966irq/58-eth023:10:143
12783996115,9cyclictest12-21ksoftirqd/023:40:130
12783996115,9cyclictest12-21ksoftirqd/023:40:120
1280499572,47cyclictest12031-21ssh23:05:122
1278399563,43cyclictest21087-21ssh23:10:130
1280499551,35cyclictest930-21snmpd23:20:122
1280499551,34cyclictest26117-21ssh21:25:142
1280499503,38cyclictest13911-21ssh22:30:112
1278399493,32cyclictest2551-21ssh00:30:120
1281599451,25cyclictest35966irq/58-eth020:25:133
1279599452,34cyclictest8498-21ssh22:45:141
1279599451,36cyclictest22924-21ssh22:35:131
1279599442,33cyclictest11695-21ssh21:35:111
1278399440,34cyclictest13752-21ssh00:00:150
12815994312,18cyclictest35966irq/58-eth023:45:123
1279599433,34cyclictest15410-21ssh21:55:121
1278399433,26cyclictest19230-21ssh22:15:130
1278399433,26cyclictest19230-21ssh22:15:120
1281599422,24cyclictest35966irq/58-eth021:20:133
1281599421,35cyclictest0-21swapper/322:55:143
1279599411,33cyclictest31515-21ssh21:10:161
12815994021,12cyclictest35966irq/58-eth023:50:143
1280499402,29cyclictest6441-21ssh21:50:122
1280499392,29cyclictest24603-21ssh23:30:132
1278399392,28cyclictest8064-21sh21:15:150
12783993912,4cyclictest12-21ksoftirqd/019:50:140
1281599381,25cyclictest0-21swapper/320:40:143
1281599371,21cyclictest35966irq/58-eth020:10:173
1281599365,20cyclictest0-21swapper/322:15:113
1281599365,20cyclictest0-21swapper/322:15:103
1280499363,25cyclictest0-21swapper/200:31:502
1279599362,5cyclictest131rcu_preempt00:00:151
1281599346,21cyclictest35966irq/58-eth000:20:123
1281599341,28cyclictest0-21swapper/322:45:123
1279599342,24cyclictest8065-21rm21:15:151
1280499333,22cyclictest0-21swapper/223:49:452
12804993317,10cyclictest7625-21rm21:30:352
1281599322,18cyclictest35966irq/58-eth023:05:123
1280499312,22cyclictest18288-21ssh21:37:332
1279599313,25cyclictest20787-21rm21:40:141
1278399310,23cyclictest17174-21ssh00:20:130
1281599304,13cyclictest35966irq/58-eth021:35:103
1281599300,24cyclictest0-21swapper/323:00:123
1280499301,23cyclictest10297-21ssh22:10:142
1280499301,11cyclictest24611-21ssh22:00:162
12815992919,4cyclictest0-21swapper/321:50:143
1281599290,25cyclictest0-21swapper/323:30:173
1281599290,24cyclictest0-21swapper/300:15:143
1280499292,20cyclictest0-21swapper/219:50:152
1280499292,18cyclictest27346irq/40-dwc2_hso19:35:152
1280499292,12cyclictest24553-21ssh00:23:402
1279599292,17cyclictest4898-21ssh22:25:131
1278399291,22cyclictest20786-21sh21:40:150
1281599283,16cyclictest35966irq/58-eth022:40:143
1281599280,22cyclictest0-21swapper/321:30:163
1280499283,19cyclictest0-21swapper/219:20:052
1280499282,21cyclictest2281-21ssh22:59:352
1280499282,19cyclictest0-21swapper/219:45:132
1280499281,11cyclictest22728-21ssh00:05:162
1278399281,21cyclictest1218-21ssh23:35:130
1281599275,17cyclictest0-21swapper/323:25:113
1281599271,21cyclictest0-21swapper/322:30:123
1281599271,20cyclictest0-21swapper/322:25:133
1281599270,24cyclictest0-21swapper/321:10:163
1281599270,23cyclictest0-21swapper/322:35:153
1280499275,15cyclictest10196-21ssh23:40:142
1280499275,15cyclictest10196-21ssh23:40:132
1280499273,17cyclictest27346irq/40-dwc2_hso20:00:222
1280499273,17cyclictest0-21swapper/200:18:342
1280499273,10cyclictest27346irq/40-dwc2_hso20:45:122
1280499272,17cyclictest27346irq/40-dwc2_hso00:10:142
1280499272,15cyclictest27346irq/40-dwc2_hso22:40:122
1280499272,11cyclictest0-21swapper/220:35:142
1280499271,18cyclictest29792-21ssh21:45:152
12783992716,5cyclictest131rcu_preempt20:40:150
1281599262,15cyclictest35966irq/58-eth021:40:153
12815992619,4cyclictest0-21swapper/322:10:133
1281599261,21cyclictest0-21swapper/319:35:153
1280499265,10cyclictest27346irq/40-dwc2_hso19:25:122
1280499262,17cyclictest27346irq/40-dwc2_hso21:00:122
1280499261,18cyclictest28178-21ssh23:50:122
1278399265,16cyclictest3057-21ssh23:00:120
12815992517,4cyclictest0-21swapper/321:15:143
12815992517,3cyclictest0-21swapper/319:20:143
1281599251,20cyclictest0-21swapper/319:25:133
1281599251,19cyclictest0-21swapper/320:30:133
1281599250,21cyclictest0-21swapper/300:35:143
1280499259,6cyclictest131rcu_preempt00:00:032
1280499259,4cyclictest27-21ksoftirqd/221:45:012
1280499259,2cyclictest131rcu_preempt23:28:412
1280499258,13cyclictest21878-21kthreadcore22:15:222
1280499258,13cyclictest21878-21kthreadcore22:15:212
1280499258,11cyclictest131rcu_preempt21:14:542
1280499254,11cyclictest27346irq/40-dwc2_hso22:50:132
1280499254,11cyclictest27346irq/40-dwc2_hso22:50:122
1281599248,13cyclictest0-21swapper/322:00:273
1281599243,15cyclictest0-21swapper/300:33:503
12815992417,3cyclictest0-21swapper/319:40:143
12815992417,3cyclictest0-21swapper/319:40:133
12815992416,3cyclictest0-21swapper/320:50:113
12815992410,11cyclictest35966irq/58-eth000:10:133
1281599240,21cyclictest0-21swapper/319:10:113
1280499249,12cyclictest27-21ksoftirqd/200:35:162
1280499247,4cyclictest27-21ksoftirqd/223:01:002
1280499244,17cyclictest27346irq/40-dwc2_hso20:30:142
1280499243,13cyclictest27346irq/40-dwc2_hso19:40:142
1280499243,13cyclictest27346irq/40-dwc2_hso19:40:142
1280499241,7cyclictest930-21snmpd19:15:142
1280499241,19cyclictest0-21swapper/219:10:142
1280499241,13cyclictest16811-21latency_hist21:20:022
12804992411,10cyclictest21302-21fschecks_time23:10:162
12804992410,7cyclictest22980-21cpuspeed22:35:142
1279599243,14cyclictest2559-21diskmemload00:30:141
1281599232,18cyclictest35966irq/58-eth022:00:013
1281599232,17cyclictest0-21swapper/320:35:123
1281599231,19cyclictest0-21swapper/323:44:383
1281599231,19cyclictest0-21swapper/323:44:373
1281599231,19cyclictest0-21swapper/321:25:133
1281599231,19cyclictest0-21swapper/321:05:283
1281599231,19cyclictest0-21swapper/320:20:133
1281599231,19cyclictest0-21swapper/320:05:133
1281599231,18cyclictest0-21swapper/319:45:133
1281599231,17cyclictest0-21swapper/300:05:173
1281599230,20cyclictest0-21swapper/321:00:133
1281599230,19cyclictest0-21swapper/323:20:163
1280499239,3cyclictest131rcu_preempt00:25:562
1280499237,6cyclictest27-21ksoftirqd/221:20:162
1280499233,6cyclictest0-21swapper/221:58:342
1280499233,12cyclictest1505-21fschecks_time22:05:172
12804992312,4cyclictest0-21swapper/223:58:582
1280499231,17cyclictest30102-21ssh23:15:142
1279599236,11cyclictest21-21ksoftirqd/122:05:051
1279599231,4cyclictest131rcu_preempt22:30:131
1278399233,17cyclictest30513-21kthreadcore23:50:210
1281599229,10cyclictest35966irq/58-eth000:00:043
1281599229,10cyclictest0-21swapper/323:55:153
1281599225,12cyclictest0-21swapper/321:45:153
12815992216,3cyclictest0-21swapper/323:15:123
1281599221,18cyclictest0-21swapper/319:30:213
1281599221,17cyclictest0-21swapper/320:15:133
1281599221,16cyclictest0-21swapper/320:45:133
1281599220,18cyclictest0-21swapper/323:35:153
1280499224,3cyclictest131rcu_preempt20:10:182
1280499222,14cyclictest27346irq/40-dwc2_hso20:25:142
1280499222,11cyclictest14308-21kthreadcore20:50:202
1280499221,5cyclictest28511-21cut22:20:162
12804992212,8cyclictest0-21swapper/219:55:162
12804992211,8cyclictest0-21swapper/222:25:142
1280499221,17cyclictest15803-21fschecks_count20:20:162
1279599227,9cyclictest21-21ksoftirqd/119:45:141
1279599221,5cyclictest6277-21sendmail-msp23:20:001
1279599221,5cyclictest0-21swapper/100:24:041
12795992213,6cyclictest0-21swapper/119:20:231
1279599221,16cyclictest32275-21find19:30:111
1279599221,14cyclictest0-21swapper/100:17:051
1279599220,18cyclictest25251-21/usr/sbin/munin20:30:281
1279599220,15cyclictest12946-21kthreadcore22:10:211
12783992214,6cyclictest0-21swapper/022:30:120
1281599211,18cyclictest0-21swapper/319:55:263
1281599211,17cyclictest0-21swapper/300:25:333
1281599210,18cyclictest0-21swapper/322:50:193
1281599210,18cyclictest0-21swapper/322:50:183
1281599210,18cyclictest0-21swapper/320:55:153
1281599210,18cyclictest0-21swapper/320:55:143
1281599210,18cyclictest0-21swapper/319:50:133
1280499212,10cyclictest2610-21fschecks_time20:40:182
1280499211,11cyclictest2218-21cut19:30:212
1280499211,11cyclictest1782-21grep23:35:192
1280499211,11cyclictest11108-21cut22:45:232
1279599215,3cyclictest131rcu_preempt23:55:201
1279599214,3cyclictest131rcu_preempt23:00:291
1279599212,16cyclictest31292-21kthreadcore20:35:211
1279599211,5cyclictest21005-21irqrtprio20:25:191
1279599211,17cyclictest7982-21kthreadcore21:50:201
1279599211,17cyclictest6611-21irqrtprio20:10:221
1279599211,17cyclictest5938-21proc_pri20:40:271
1279599211,17cyclictest21760-21ls21:00:151
12795992111,3cyclictest21-21ksoftirqd/120:55:231
12795992111,3cyclictest21-21ksoftirqd/120:55:221
1278399213,15cyclictest10206-21ps21:50:280
1278399212,2cyclictest131rcu_preempt23:30:130
1278399212,14cyclictest31062-21diskmemload23:08:360
1278399211,17cyclictest22616-21apt-get19:20:130
1281599200,17cyclictest0-21swapper/322:20:133
1281599200,17cyclictest0-21swapper/322:05:123
1280499207,3cyclictest27-21ksoftirqd/220:15:182
1280499202,10cyclictest17788-21kthreadcore20:55:222
1280499202,10cyclictest17788-21kthreadcore20:55:212
1280499201,10cyclictest4636-21grep20:05:282
1279599203,2cyclictest131rcu_preempt23:10:161
1279599202,5cyclictest25402-21ssh23:47:101
1279599202,3cyclictest131rcu_preempt00:25:591
1279599202,16cyclictest26447-21apt-get22:55:121
1279599202,12cyclictest11594-21ssh00:35:141
1279599201,3cyclictest0-21swapper/120:50:231
12795992012,5cyclictest6574-21sendmail_mailtr21:30:301
1279599201,16cyclictest6915-21fschecks_time23:20:171
1279599201,16cyclictest30313-21kthreadcore20:00:271
1279599201,16cyclictest19755-21ssh23:25:281
1279599201,16cyclictest12622-21kthreadcore20:15:211
12795992010,7cyclictest28166-21apt-get23:50:131
1278399203,15cyclictest12352-21kthreadcore22:10:220
1278399203,14cyclictest31181-21kthreadcore23:15:210
1278399203,14cyclictest22168-21kthreadcore20:25:200
1278399202,14cyclictest8499-21sh22:45:140
1278399201,7cyclictest8195-21latency_hist22:45:020
1278399201,5cyclictest17475-21munin-run19:15:020
12783992012,5cyclictest0-21swapper/000:35:180
1278399201,17cyclictest24588-21find22:00:150
12783992011,6cyclictest0-21swapper/022:55:130
1278399201,15cyclictest26099-21ssh00:25:170
1278399200,3cyclictest131rcu_preempt21:55:100
1281599191,15cyclictest0-21swapper/319:15:153
1281599190,16cyclictest0-21swapper/320:00:213
1280499191,11cyclictest28924-21kthreadcore21:05:212
1280499191,10cyclictest12899-21grep19:10:012
1279599197,9cyclictest21-21ksoftirqd/122:20:221
1279599192,2cyclictest131rcu_preempt00:13:161
1279599192,14cyclictest17332-21grep21:20:161
1279599191,4cyclictest0-21swapper/121:26:121
1279599191,4cyclictest0-21swapper/119:40:171
1279599191,4cyclictest0-21swapper/119:40:171
1279599191,3cyclictest0-21swapper/122:02:261
1279599191,16cyclictest23605-21kthreadcore00:05:241
1279599191,16cyclictest22134-21kthreadcore22:15:221
1279599191,16cyclictest22134-21kthreadcore22:15:211
1279599191,15cyclictest9237-21kthreadcore20:45:211
1279599191,15cyclictest5222-21timerwakeupswit23:35:301
1279599191,15cyclictest30304-21kthreadcore19:25:221
1279599191,15cyclictest26472-21acpi21:05:101
1279599191,15cyclictest21155-21kthreadcore19:50:211
1279599190,3cyclictest0-21swapper/123:40:041
1279599190,3cyclictest0-21swapper/123:40:031
1278399194,3cyclictest131rcu_preempt21:00:130
1278399193,5cyclictest26453-21sh22:35:230
1278399193,14cyclictest18313-21tr20:20:220
1278399193,14cyclictest1302-21awk21:45:290
1278399193,13cyclictest5132-21mailstats22:05:290
1278399193,13cyclictest32453-21kthreadcore00:10:190
1278399193,13cyclictest23327-21unixbench_singl23:45:300
1278399192,3cyclictest131rcu_preempt22:25:130
1278399192,13cyclictest26106-21apt-get21:25:110
12783991911,5cyclictest32403-21unixbench_multi22:20:310
12783991911,5cyclictest23481-21kthreadcore00:05:230
1278399191,14cyclictest11809-21ssh00:15:240
12783991910,6cyclictest2196-21cron20:40:000
1279599188,7cyclictest4673-21apt-get19:35:151
1279599183,13cyclictest21527-21unixbench_multi22:50:291
1279599183,13cyclictest21527-21unixbench_multi22:50:281
1279599182,13cyclictest15696-21apt-get20:20:121
1279599181,15cyclictest25959-21kthreadcore19:55:231
1279599181,15cyclictest1285-21fschecks_time20:05:181
1279599181,14cyclictest3594-21tail22:40:301
1279599181,14cyclictest1498-21unixbench_singl21:45:311
1278399186,3cyclictest131rcu_preempt23:25:140
1278399185,9cyclictest0-21swapper/021:10:180
1278399183,13cyclictest5062-21ssh23:55:180
1278399183,13cyclictest4448-21awk19:35:010
1278399183,13cyclictest31976-21apt-get19:30:010
1278399183,13cyclictest21449-21cron21:00:010
1278399183,13cyclictest21449-21cron21:00:000
1278399183,13cyclictest17451-21sort21:20:170
1278399183,13cyclictest15360-21ntpq21:35:230
1278399183,13cyclictest10382-21/usr/sbin/munin23:20:260
1278399183,12cyclictest7430-21tune2fs20:45:160
1278399183,12cyclictest5701-21apt-get20:10:010
1278399183,12cyclictest20870-21cut19:15:230
1278399183,12cyclictest10954-21fschecks_time20:15:160
1279599172,12cyclictest13123-21tune2fs19:10:151
1279599171,3cyclictest0-21swapper/123:30:121
1279599170,14cyclictest12134-21cpuspeed23:05:131
1278399178,7cyclictest6102-21df_inode20:10:190
1278399178,7cyclictest16615-21cron20:55:010
1278399177,7cyclictest29261-21irqrtprio20:00:270
1278399174,10cyclictest25963-21kthreadcore19:55:220
1278399173,12cyclictest28206-21kthreadcore20:30:220
1278399173,12cyclictest26587-21fschecks_count21:05:140
1278399173,12cyclictest19101-21latency_hist19:50:010
1278399173,12cyclictest17799-21if_eth022:50:170
1278399173,12cyclictest17799-21if_eth022:50:160
1278399172,5cyclictest131rcu_preempt21:30:240
1278399171,3cyclictest0-21swapper/019:40:190
1278399171,3cyclictest0-21swapper/019:40:180
1279599160,14cyclictest17692-21find19:15:141
1278399163,10cyclictest6571-21kthreadcore19:35:230
1278399153,9cyclictest12898-21latency_hist19:10:010
1281599141,10cyclictest0-21swapper/319:10:013
1279599146,5cyclictest0-21swapper/119:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional