You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-19 - 22:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Mar 19, 2026 12:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2552199675,50cyclictest14716-21ssh11:00:110
2553799554,6cyclictest131rcu_preempt12:15:111
2552199541,36cyclictest27790-21ssh09:55:110
2553799522,39cyclictest15317-21ssh09:30:111
25548995110,18cyclictest27-21ksoftirqd/212:30:112
2553799517,9cyclictest131rcu_preempt07:30:101
2553799515,8cyclictest21-21ksoftirqd/108:40:131
2552199512,38cyclictest9940-21ssh09:45:140
2553799503,5cyclictest131rcu_preempt09:55:131
2553799503,36cyclictest27244-21ssh11:25:121
2553799491,39cyclictest12856-21ssh11:35:111
2553799484,24cyclictest24347-21ssh09:35:111
2553799465,28cyclictest11273-21diskmemload10:00:131
2553799451,32cyclictest18910-21ssh09:50:131
2554899445,19cyclictest27-21ksoftirqd/211:40:092
2552199432,32cyclictest9221-21ssh11:15:130
25521994313,5cyclictest131rcu_preempt09:10:120
2552199431,32cyclictest6352-21ssh09:25:120
2553799422,31cyclictest5211-21ssh12:25:111
25537994017,10cyclictest21-21ksoftirqd/110:55:111
2553799392,28cyclictest3872-21ssh11:30:121
25537993912,19cyclictest21-21ksoftirqd/110:30:131
25537993821,7cyclictest21-21ksoftirqd/109:40:131
2553799373,25cyclictest10831-21ssh12:10:141
25521993710,13cyclictest10815-21ssh12:10:120
2555899365,20cyclictest28366irq/58-eth011:00:113
25548993611,21cyclictest27-21ksoftirqd/210:27:372
2553799362,31cyclictest16915-21ssh10:25:141
2553799361,29cyclictest0-21swapper/112:00:141
2552199362,24cyclictest16890-21ssh10:25:110
2555899351,21cyclictest28366irq/58-eth009:25:133
2554899350,26cyclictest0-21swapper/207:10:142
25537993528,3cyclictest0-21swapper/111:45:131
2555899345,15cyclictest28366irq/58-eth011:10:113
25548993415,11cyclictest0-21swapper/207:25:122
2554899341,28cyclictest16437-21ssh11:55:132
2552199349,2cyclictest131rcu_preempt08:50:140
25548993318,10cyclictest0-21swapper/211:10:112
2554899331,26cyclictest9931-21ssh09:45:132
2553799332,3cyclictest0-21swapper/111:55:121
2553799332,25cyclictest21852-21ssh11:40:111
25537993314,15cyclictest21-21ksoftirqd/110:35:141
25537993312,16cyclictest21-21ksoftirqd/107:40:111
2554899327,20cyclictest20722-21ssh09:15:122
2554899323,15cyclictest22346irq/40-dwc2_hso08:45:122
2554899322,18cyclictest31404-21ssh10:15:132
2553799311,20cyclictest21-21ksoftirqd/109:10:121
2555899301,15cyclictest28366irq/58-eth011:15:123
2554899303,21cyclictest0-21swapper/210:32:372
2554899302,21cyclictest16403-21ssh10:40:482
25558992911,11cyclictest28366irq/58-eth011:40:113
2554899294,17cyclictest22346irq/40-dwc2_hso12:20:102
2554899292,20cyclictest0-21swapper/208:20:132
2552199291,21cyclictest30865-21ssh11:45:140
2555899283,14cyclictest0-21swapper/311:55:133
2554899288,8cyclictest27-21ksoftirqd/209:11:062
2554899286,18cyclictest22346irq/40-dwc2_hso12:10:112
25537992811,12cyclictest21-21ksoftirqd/108:10:121
2554899278,11cyclictest27-21ksoftirqd/212:09:592
2554899277,15cyclictest14729-21sh11:00:132
2554899275,13cyclictest22346irq/40-dwc2_hso07:45:132
2554899275,10cyclictest27-21ksoftirqd/211:52:552
2554899272,19cyclictest25224-21ssh10:45:432
2554899272,18cyclictest22346irq/40-dwc2_hso08:00:112
2554899272,18cyclictest22346irq/40-dwc2_hso07:50:122
2554899272,11cyclictest0-21swapper/207:55:122
25537992711,12cyclictest768-21sed10:50:291
2552199272,19cyclictest32507-21sh10:32:370
25521992713,11cyclictest20893-21ssh12:15:180
25521992712,8cyclictest12-21ksoftirqd/009:35:270
2555899269,9cyclictest28366irq/58-eth010:15:133
2554899267,3cyclictest131rcu_preempt09:50:152
2554899265,8cyclictest27-21ksoftirqd/210:35:022
2554899264,13cyclictest22346irq/40-dwc2_hso08:05:112
2554899263,14cyclictest22346irq/40-dwc2_hso08:50:122
2553799265,11cyclictest0-21swapper/111:20:131
2553799261,17cyclictest31379-21ssh10:15:121
25521992610,9cyclictest12-21ksoftirqd/009:20:120
25521992610,9cyclictest12-21ksoftirqd/009:20:120
2555899252,17cyclictest0-21swapper/312:05:103
2554899256,12cyclictest27-21ksoftirqd/211:09:282
2554899254,6cyclictest131rcu_preempt11:45:242
2554899254,17cyclictest261rcuc/210:01:112
2554899254,13cyclictest131rcu_preempt09:40:032
25548992515,8cyclictest261rcuc/211:38:462
2554899251,17cyclictest18942-21sh09:30:232
2554899251,15cyclictest30093-21ls09:20:172
2554899251,15cyclictest30093-21ls09:20:172
2552199258,12cyclictest12-21ksoftirqd/010:42:370
2552199257,10cyclictest12-21ksoftirqd/011:33:400
25521992516,6cyclictest6383-21ssh10:35:260
2552199251,19cyclictest22357-21ssh10:10:120
25521992511,4cyclictest12-21ksoftirqd/010:50:230
25558992417,5cyclictest28366irq/58-eth010:35:113
25558992416,5cyclictest28366irq/58-eth009:15:133
2554899247,11cyclictest27-21ksoftirqd/210:08:012
2554899246,11cyclictest27-21ksoftirqd/212:38:462
2554899245,12cyclictest27-21ksoftirqd/209:25:052
2554899244,8cyclictest27-21ksoftirqd/210:54:512
2554899243,8cyclictest27-21ksoftirqd/209:58:472
2554899243,18cyclictest7197-21ssh12:25:202
2554899243,18cyclictest5752-21ssh10:55:122
2554899243,14cyclictest27-21ksoftirqd/212:15:232
2554899242,17cyclictest7935-21ssh10:20:132
25548992418,4cyclictest261rcuc/211:25:272
25548992417,5cyclictest27-21ksoftirqd/212:00:132
2554899241,12cyclictest1676-21kthreadcore09:40:192
2553799243,11cyclictest21-21ksoftirqd/109:00:131
2553799242,19cyclictest13430-21ssh10:05:131
25537992416,5cyclictest0-21swapper/109:15:031
25537992413,5cyclictest23754-21sh11:05:121
2552199248,8cyclictest0-21swapper/011:58:340
2552199241,17cyclictest7473-21ssh11:50:130
25521992411,7cyclictest0-21swapper/012:33:460
2555899232,10cyclictest0-21swapper/309:30:123
2554899238,11cyclictest6759-21cut11:30:202
2554899235,3cyclictest131rcu_preempt09:10:012
2554899235,12cyclictest23409-21/usr/sbin/munin08:15:122
2554899233,6cyclictest27-21ksoftirqd/207:35:012
2554899232,13cyclictest18127-21kthreadcore08:40:212
2554899232,12cyclictest22346irq/40-dwc2_hso08:25:122
25548992313,8cyclictest17837-21sendmail11:20:012
25548992312,8cyclictest261rcuc/211:22:462
25548992312,7cyclictest0-21swapper/208:30:182
25548992312,7cyclictest0-21swapper/208:30:172
2553799232,16cyclictest23041-21irqstats10:10:161
2553799232,15cyclictest22977-21ssh12:35:121
2552199237,7cyclictest12-21ksoftirqd/007:10:170
2552199232,15cyclictest15337-21ssh09:30:120
25521992313,3cyclictest0-21swapper/011:24:320
25521992312,3cyclictest0-21swapper/010:20:430
25521992310,7cyclictest0-21swapper/012:20:060
25521992310,3cyclictest12-21ksoftirqd/011:05:270
25521992310,3cyclictest0-21swapper/010:16:020
2555899229,9cyclictest28366irq/58-eth011:25:143
25558992213,6cyclictest0-21swapper/310:00:173
25558992213,6cyclictest0-21swapper/309:37:303
25558992210,9cyclictest0-21swapper/311:21:173
25558992210,9cyclictest0-21swapper/310:55:213
25558992210,9cyclictest0-21swapper/310:30:293
2554899223,12cyclictest22346irq/40-dwc2_hso07:15:122
25548992211,8cyclictest24260-21sh10:10:192
2554899221,10cyclictest21184-21tail07:35:282
2554899220,15cyclictest0-21swapper/207:20:122
2553799221,15cyclictest7929-21ssh10:20:121
2552199229,7cyclictest0-21swapper/010:09:470
2552199228,8cyclictest0-21swapper/008:05:180
2552199227,3cyclictest0-21swapper/009:54:500
2552199225,8cyclictest12-21ksoftirqd/010:48:570
2552199223,8cyclictest12-21ksoftirqd/010:55:390
25521992210,6cyclictest8914-21ssh12:08:090
25521992210,2cyclictest12-21ksoftirqd/009:10:010
2555899213,10cyclictest28366irq/58-eth007:50:113
25558992115,4cyclictest0-21swapper/310:23:563
25558992115,4cyclictest0-21swapper/309:55:433
25558992115,4cyclictest0-21swapper/309:05:253
25558992115,3cyclictest321rcuc/308:50:233
25558992114,4cyclictest0-21swapper/310:40:083
25558992112,7cyclictest0-21swapper/309:10:423
25558992112,6cyclictest0-21swapper/312:35:143
2554899213,10cyclictest11352-21kthreadcore08:35:192
2554899212,8cyclictest22587-21irqcore07:40:172
2554899211,16cyclictest2389-21tr09:00:192
2554899211,16cyclictest18823-21hwlatdetect08:10:162
2553799211,6cyclictest18488-21ps11:00:251
25537992112,6cyclictest20021-21sh10:45:021
2553799211,15cyclictest7846-21cut07:25:161
2553799211,15cyclictest27069-21cut07:45:161
2552199219,3cyclictest12-21ksoftirqd/011:35:120
2552199217,7cyclictest12-21ksoftirqd/008:26:360
2552199216,7cyclictest12-21ksoftirqd/009:19:040
2552199216,7cyclictest0-21swapper/011:29:410
2552199216,4cyclictest12-21ksoftirqd/010:00:180
2552199213,3cyclictest131rcu_preempt11:40:170
25521992112,3cyclictest0-21swapper/011:11:370
2552199211,15cyclictest12247-21ls08:00:210
25521992110,3cyclictest12-21ksoftirqd/012:35:490
25521992110,3cyclictest12-21ksoftirqd/007:50:140
2555899207,9cyclictest0-21swapper/309:50:113
25558992014,4cyclictest0-21swapper/312:30:143
25558992014,4cyclictest0-21swapper/312:15:113
25558992014,4cyclictest0-21swapper/311:45:263
25558992014,4cyclictest0-21swapper/311:35:143
25558992014,4cyclictest0-21swapper/310:53:243
25558992014,4cyclictest0-21swapper/310:45:183
25558992014,4cyclictest0-21swapper/310:27:543
25558992014,4cyclictest0-21swapper/310:13:453
25558992014,4cyclictest0-21swapper/310:05:313
25558992014,4cyclictest0-21swapper/309:46:363
25558992014,4cyclictest0-21swapper/308:10:203
25558992013,5cyclictest0-21swapper/312:25:123
25558992012,6cyclictest0-21swapper/312:20:053
2554899201,10cyclictest29468-21cpu08:55:152
2553799202,13cyclictest25973-21ssh10:46:371
25537992012,5cyclictest587-21usage_Anybus12:20:291
2553799201,15cyclictest7472-21sh11:50:141
2552199208,6cyclictest131rcu_preempt09:41:170
2552199207,8cyclictest12-21ksoftirqd/012:27:120
2552199207,8cyclictest12-21ksoftirqd/007:23:010
2552199203,8cyclictest750-21sendmail-mta09:03:020
2552199202,5cyclictest12-21ksoftirqd/012:00:220
2552199201,13cyclictest10397-21cut08:35:160
25521992010,7cyclictest11531-21grep07:25:280
2555899193,11cyclictest0-21swapper/308:40:133
25558991914,3cyclictest0-21swapper/309:41:073
25558991913,4cyclictest0-21swapper/312:11:053
25558991913,4cyclictest0-21swapper/312:00:333
25558991913,4cyclictest0-21swapper/311:50:183
25558991913,4cyclictest0-21swapper/311:30:113
25558991913,4cyclictest0-21swapper/311:05:073
25558991913,4cyclictest0-21swapper/309:20:273
25558991913,4cyclictest0-21swapper/309:20:273
25558991913,4cyclictest0-21swapper/307:45:153
25558991913,4cyclictest0-21swapper/307:40:143
25558991913,4cyclictest0-21swapper/307:30:183
2553799199,7cyclictest2877-21cut12:05:171
2553799194,10cyclictest17014-21kworker/1:207:35:121
2553799191,8cyclictest4805-21usage_EthcTask11:10:301
25537991911,5cyclictest14104-21/usr/sbin/munin12:30:101
25537991911,5cyclictest10490-21usage_Anybus09:25:291
25537991910,6cyclictest12831-21grep08:00:271
2552199198,6cyclictest131rcu_preempt07:33:010
2552199197,2cyclictest131rcu_preempt08:45:280
2552199195,6cyclictest12-21ksoftirqd/008:20:150
2552199193,4cyclictest12-21ksoftirqd/008:35:010
2552199193,4cyclictest12-21ksoftirqd/008:35:010
25521991910,6cyclictest0-21swapper/007:55:280
2552199190,5cyclictest12-21ksoftirqd/009:00:010
2555899182,10cyclictest0-21swapper/307:55:123
2555899182,10cyclictest0-21swapper/307:20:123
25558991813,3cyclictest0-21swapper/308:00:183
25558991812,4cyclictest321rcuc/308:35:373
25558991812,4cyclictest0-21swapper/308:30:233
25558991812,4cyclictest0-21swapper/308:30:233
25558991812,4cyclictest0-21swapper/308:05:133
25558991812,4cyclictest0-21swapper/307:35:293
25558991811,5cyclictest33-21ksoftirqd/308:15:143
2554899181,8cyclictest25616-21latency_hist07:10:022
2553799188,7cyclictest1881-21kthreadcore08:25:191
2553799183,2cyclictest131rcu_preempt11:18:531
2553799182,13cyclictest32645-21kthreadcore09:20:231
2553799182,13cyclictest32645-21kthreadcore09:20:221
2553799181,9cyclictest10541-21unixbench_singl09:05:301
2553799181,13cyclictest3529-21sshd07:20:171
2552199188,2cyclictest12-21ksoftirqd/008:10:200
2552199183,10cyclictest0-21swapper/007:35:050
25521991812,4cyclictest0-21swapper/008:15:150
25521991812,4cyclictest0-21swapper/007:45:140
25521991812,4cyclictest0-21swapper/007:40:220
25521991812,4cyclictest0-21swapper/007:15:120
2555899174,7cyclictest0-21swapper/308:20:103
25558991712,3cyclictest0-21swapper/307:10:263
25558991711,4cyclictest0-21swapper/309:00:243
25558991711,4cyclictest0-21swapper/308:25:163
25558991711,4cyclictest0-21swapper/307:25:203
25558991711,4cyclictest0-21swapper/307:15:153
2553799176,8cyclictest30826-21irqcore07:15:181
2553799172,13cyclictest2061-21kthreadcore07:50:201
2553799172,11cyclictest10182-21perl09:45:171
2553799171,12cyclictest9035-21wc08:30:261
2553799171,12cyclictest9035-21wc08:30:261
2553799170,9cyclictest28357-21cut08:20:141
2553799170,7cyclictest24826-21fschecks_time08:50:151
2553799170,10cyclictest10215-21apt-get08:35:121
2552199171,14cyclictest15522-21grep08:40:180
25558991611,3cyclictest0-21swapper/308:45:293
2553799167,7cyclictest21-21ksoftirqd/107:55:121
2553799163,11cyclictest27520-21kthreadcore07:10:221
2553799162,9cyclictest0-21swapper/108:55:151
2553799162,11cyclictest24055-21sed08:15:171
2553799162,11cyclictest22122-21kthreadcore08:45:201
2553799160,9cyclictest17546-21ls08:05:271
2555899142,10cyclictest0-21swapper/308:55:153
2553799141,10cyclictest25568-21latency_hist07:10:021
2552199140,10cyclictest25524-21latency_hist07:10:020
2555899111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional