You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 20:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sat May 30, 2026 12:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28772996044,10cyclictest35966irq/58-eth009:55:143
2876399581,48cyclictest19366-21ssh11:00:132
2876399581,43cyclictest17163-21ssh10:05:122
28772994626,13cyclictest35966irq/58-eth010:05:133
2876399424,27cyclictest27-21ksoftirqd/211:20:112
2877299413,25cyclictest35966irq/58-eth011:55:113
28772994131,6cyclictest0-21swapper/311:25:103
2875199392,3cyclictest131rcu_preempt08:25:121
2875199392,3cyclictest131rcu_preempt08:25:121
2875199392,29cyclictest18584-21ssh09:30:131
2877299383,19cyclictest35966irq/58-eth010:40:133
2875199380,3cyclictest131rcu_preempt12:20:121
2877299374,22cyclictest35966irq/58-eth011:40:123
2877299371,22cyclictest35966irq/58-eth008:45:123
2876399371,23cyclictest28426-21ssh12:35:112
2873999372,24cyclictest12006-21ssh10:20:160
2877299363,20cyclictest35966irq/58-eth012:25:123
2876399351,2cyclictest131rcu_preempt09:20:162
28763993510,15cyclictest0-21swapper/211:40:132
2877299344,18cyclictest35966irq/58-eth011:15:143
2877299342,19cyclictest35966irq/58-eth010:15:153
2876399341,21cyclictest21081-21ssh11:55:122
2877299334,20cyclictest35966irq/58-eth009:50:133
2877299333,23cyclictest35966irq/58-eth011:50:153
2877299333,20cyclictest35966irq/58-eth011:35:153
2877299333,20cyclictest35966irq/58-eth011:35:153
2877299326,21cyclictest35966irq/58-eth011:20:123
2877299322,16cyclictest35966irq/58-eth007:15:113
2876399312,22cyclictest0-21swapper/212:00:132
2876399311,21cyclictest4933-21ssh11:10:122
2876399311,13cyclictest14734-21ssh09:10:152
2876399292,20cyclictest0-21swapper/208:05:162
2876399284,7cyclictest0-21swapper/212:20:132
2876399282,11cyclictest0-21swapper/209:40:132
2876399282,10cyclictest19474-21ssh12:30:132
2877299272,11cyclictest35966irq/58-eth008:20:123
28772992719,3cyclictest0-21swapper/310:30:103
2876399272,19cyclictest0-21swapper/208:15:132
2876399272,19cyclictest0-21swapper/207:45:132
2876399271,7cyclictest0-21swapper/208:25:122
2876399271,7cyclictest0-21swapper/208:25:122
2876399271,13cyclictest10495-21ssh12:25:142
2876399270,10cyclictest8102-21ssh10:00:122
28772992616,6cyclictest0-21swapper/310:35:153
28763992612,11cyclictest4689-21cron09:00:022
2875199264,6cyclictest131rcu_preempt12:25:131
2873999260,2cyclictest131rcu_preempt11:50:140
2877299259,11cyclictest0-21swapper/309:10:153
2877299252,11cyclictest35966irq/58-eth009:30:133
28772992515,5cyclictest0-21swapper/312:05:133
28772992512,9cyclictest0-21swapper/309:20:133
2877299251,14cyclictest35966irq/58-eth009:25:133
2877299251,14cyclictest35966irq/58-eth009:25:123
2876399252,9cyclictest0-21swapper/208:45:132
28763992512,7cyclictest27-21ksoftirqd/209:35:142
28763992512,6cyclictest19742-21ssh11:16:382
2876399251,21cyclictest21033-21cpuspeed10:25:142
2876399251,16cyclictest0-21swapper/209:05:132
2873999251,20cyclictest23102-21ssh11:00:260
28772992417,3cyclictest0-21swapper/310:45:123
28772992417,3cyclictest0-21swapper/310:45:123
28772992414,6cyclictest35966irq/58-eth009:15:153
28772992414,6cyclictest35966irq/58-eth009:15:153
28772992413,8cyclictest0-21swapper/310:10:153
28772992412,9cyclictest0-21swapper/310:25:123
28772992411,9cyclictest0-21swapper/312:15:123
2876399246,15cyclictest0-21swapper/212:05:282
2876399242,15cyclictest0-21swapper/209:55:132
2876399242,13cyclictest11991-21ssh10:20:152
2875199242,19cyclictest417-21ssh09:20:151
2875199241,17cyclictest14312-21diskmemload09:30:021
2875199241,17cyclictest14312-21diskmemload09:30:021
2875199240,3cyclictest131rcu_preempt09:45:141
2873999242,18cyclictest22309-21ssh09:50:140
2877299236,11cyclictest0-21swapper/312:00:123
2877299232,11cyclictest35966irq/58-eth012:20:133
28772992318,3cyclictest0-21swapper/308:25:123
28772992318,3cyclictest0-21swapper/308:25:113
28772992317,4cyclictest0-21swapper/312:30:113
28772992317,4cyclictest0-21swapper/309:00:103
28772992316,5cyclictest0-21swapper/307:35:113
28772992315,5cyclictest0-21swapper/311:10:133
28772992315,3cyclictest0-21swapper/311:45:113
28772992315,3cyclictest0-21swapper/309:45:133
2876399234,12cyclictest27346irq/40-dwc2_hso07:40:122
2876399233,14cyclictest131rcu_preempt10:40:302
2876399232,6cyclictest0-21swapper/212:15:052
28763992312,9cyclictest26751-21ssh09:34:342
2876399231,19cyclictest9445-21ssh09:25:142
2876399231,19cyclictest9445-21ssh09:25:142
2876399231,12cyclictest14312-21diskmemload09:15:152
2876399231,12cyclictest14312-21diskmemload09:15:142
2876399231,11cyclictest930-21snmpd11:35:152
2876399231,11cyclictest930-21snmpd11:35:152
2875199232,16cyclictest31390-21ssh09:55:141
2875199231,7cyclictest13058-21usage_EthcTask11:30:311
28751992312,4cyclictest14312-21diskmemload12:17:001
2873999232,17cyclictest1386-21rm10:50:150
28739992318,3cyclictest0-21swapper/007:55:140
28739992312,7cyclictest26200-21sh10:10:150
2877299223,11cyclictest0-21swapper/308:00:133
2877299221,5cyclictest0-21swapper/312:10:133
28772992215,4cyclictest0-21swapper/312:35:123
28772992215,3cyclictest0-21swapper/309:05:133
28772992214,4cyclictest0-21swapper/311:30:143
28772992212,7cyclictest0-21swapper/307:20:133
28772992212,7cyclictest0-21swapper/307:20:133
28772992210,9cyclictest0-21swapper/309:42:593
2876399226,14cyclictest24141-21cut09:50:212
28763992211,9cyclictest9512-21ssh10:17:352
28763992211,9cyclictest18568-21ssh11:52:272
2876399221,17cyclictest4992-21ssh10:50:262
2876399221,16cyclictest12170-21ssh11:30:252
2876399221,12cyclictest28632-21tune2fs11:05:182
2875199225,7cyclictest131rcu_preempt10:05:131
2875199225,3cyclictest131rcu_preempt11:15:011
2875199222,5cyclictest53646usb-storage12:10:041
28751992215,4cyclictest14312-21diskmemload10:50:151
2873999222,14cyclictest14312-21diskmemload10:55:150
28739992214,5cyclictest536-21usb-storage10:00:110
28739992211,8cyclictest4163-21sh09:40:130
2873999221,17cyclictest2876-21ssh10:15:140
28772992116,2cyclictest0-21swapper/311:05:143
28772992115,3cyclictest0-21swapper/307:30:143
28772992114,3cyclictest0-21swapper/310:20:153
28772992114,3cyclictest0-21swapper/307:45:123
28772992113,3cyclictest0-21swapper/308:05:133
2876399218,6cyclictest7329-21apt-get07:55:142
2876399218,10cyclictest13267-21sshd10:37:332
2876399214,5cyclictest6417-21irqcore07:20:182
2876399214,5cyclictest6417-21irqcore07:20:182
2876399211,15cyclictest16831-21latency_hist08:05:002
2876399210,17cyclictest1109-21ls10:50:012
2876399210,17cyclictest1109-21ls10:50:002
2875199215,4cyclictest27765-21tr10:10:211
28751992112,6cyclictest14697-21sort09:10:131
2873999215,3cyclictest131rcu_preempt09:05:130
28739992114,4cyclictest0-21swapper/011:05:150
28739992113,5cyclictest28445-21kthreadcore11:40:200
28739992111,6cyclictest12-21ksoftirqd/012:20:130
28772992015,2cyclictest0-21swapper/308:35:123
28772992010,7cyclictest0-21swapper/309:35:223
2876399208,5cyclictest0-21swapper/208:50:172
2876399205,6cyclictest27389-21ssh10:10:212
2876399204,13cyclictest9517-21cron09:05:012
2876399201,9cyclictest3123-21ssh11:45:122
2876399201,16cyclictest30582-21irqrtprio10:30:162
2876399201,16cyclictest24425-21apt12:15:012
2876399201,15cyclictest14088-21ssh10:55:282
2876399200,11cyclictest485166ssh11:25:592
2875199203,3cyclictest131rcu_preempt12:30:301
2875199202,9cyclictest0-21swapper/112:06:521
28751992014,4cyclictest0-21swapper/112:05:021
2875199201,16cyclictest32001-21ssh11:25:131
28751992011,6cyclictest17941-1kworker/u9:210:35:191
28751992011,6cyclictest17889-21cron08:40:011
2875199201,14cyclictest27551-21ssh09:35:151
2873999208,9cyclictest12-21ksoftirqd/008:35:140
2873999207,4cyclictest131rcu_preempt12:30:290
2873999207,10cyclictest9418-21ssh09:25:120
2873999207,10cyclictest9418-21ssh09:25:120
2873999206,9cyclictest614-21dbus-daemon07:25:120
2873999203,5cyclictest12-21ksoftirqd/009:10:280
2873999202,3cyclictest131rcu_preempt09:30:230
28739992013,4cyclictest0-21swapper/011:10:130
28739992011,7cyclictest5545-21sshd09:57:330
2873999201,16cyclictest30049-21apt-get12:00:120
2873999201,15cyclictest18142-21apt-get08:40:110
2877299194,8cyclictest35966irq/58-eth011:00:133
2877299192,14cyclictest0-21swapper/310:50:203
2877299192,10cyclictest0-21swapper/307:50:043
28772991914,2cyclictest0-21swapper/310:55:153
2877299191,13cyclictest0-21swapper/308:55:223
2876399199,8cyclictest0-21swapper/207:50:042
2876399191,9cyclictest28998-21fschecks_count07:10:162
2876399191,15cyclictest4968-21wc07:15:282
2876399191,15cyclictest17052-21wc09:45:282
2876399191,13cyclictest0-21swapper/208:40:122
2876399191,11cyclictest27346irq/40-dwc2_hso07:30:132
2876399191,10cyclictest8611-21ls08:30:142
2876399191,10cyclictest14900-21kthreadcore08:35:202
2875199199,7cyclictest8735-21cat09:40:291
2875199198,9cyclictest1332-21df07:15:151
2875199198,3cyclictest131rcu_preempt11:45:121
2875199193,3cyclictest131rcu_preempt10:45:171
2875199193,3cyclictest131rcu_preempt10:45:171
2875199191,7cyclictest4705-21munin-run09:00:021
28751991913,4cyclictest0-21swapper/108:10:281
28751991911,5cyclictest710-21cron10:25:011
2875199191,15cyclictest2887-21ssh10:15:151
28751991910,7cyclictest31071-21cut12:35:211
28751991910,6cyclictest31148-21basename08:20:011
2873999199,7cyclictest630-21Async09:36:310
2873999196,3cyclictest131rcu_preempt11:45:130
2873999193,12cyclictest27429-21ssh09:15:260
2873999193,12cyclictest27429-21ssh09:15:260
2873999192,13cyclictest135-21systemd-journal12:15:010
2873999191,8cyclictest29983-21cut12:35:200
28739991915,2cyclictest0-21swapper/007:35:110
28739991913,4cyclictest415-21sh09:20:150
28739991912,4cyclictest24742-21ssh10:45:130
28739991912,4cyclictest24742-21ssh10:45:130
2873999191,13cyclictest12-21ksoftirqd/011:25:110
2873999191,11cyclictest20283-21ls08:05:250
28739991910,7cyclictest20980-21sh10:25:130
28739991910,6cyclictest26127-21cron11:40:010
28739991910,6cyclictest26127-21cron11:40:000
28739991910,6cyclictest10775-21cut11:30:210
2873999190,12cyclictest20213-21latency_hist07:35:010
2877299189,7cyclictest0-21swapper/307:40:283
2877299182,11cyclictest0-21swapper/310:00:053
2877299181,11cyclictest0-21swapper/307:25:173
2876399187,4cyclictest31722-21irqcore08:20:172
2876399186,3cyclictest21771-21cpu08:10:142
2876399181,8cyclictest20448-21cpu07:35:132
2876399181,8cyclictest11892-21kthreadcore07:25:202
2875199189,7cyclictest28692-21sshd07:40:231
2875199181,9cyclictest14312-21diskmemload11:21:301
2875199181,9cyclictest0-21swapper/111:44:211
2875199181,9cyclictest0-21swapper/111:00:041
2875199181,9cyclictest0-21swapper/110:40:581
2875199181,4cyclictest0-21swapper/111:37:131
2875199181,4cyclictest0-21swapper/111:37:121
2875199181,13cyclictest30013-21ssh10:30:111
2873999189,6cyclictest26158-21cut07:40:170
2873999189,6cyclictest0-21swapper/011:55:260
2873999184,10cyclictest12-21ksoftirqd/010:35:150
2873999183,12cyclictest17180-21ssh10:05:140
2873999182,3cyclictest131rcu_preempt12:26:580
2873999182,13cyclictest930-21snmpd09:48:080
2873999182,13cyclictest5822-21cron07:20:010
2873999182,13cyclictest3893-21timerwakeupswit08:55:280
2873999182,12cyclictest19437-21ntp_states10:40:260
2873999181,7cyclictest2504-21latency_hist07:50:020
28739991812,4cyclictest26894-21timerandwakeup11:20:280
28739991812,3cyclictest0-21swapper/011:15:130
2873999181,13cyclictest24886-21sh12:15:130
28739991810,6cyclictest3334-21sshd10:31:320
2877299178,6cyclictest0-21swapper/307:10:193
2877299172,9cyclictest0-21swapper/308:50:143
2877299172,10cyclictest0-21swapper/308:15:133
28772991712,3cyclictest0-21swapper/308:40:123
2875199179,6cyclictest11716-21ntp_192.168.11410:00:231
2875199179,5cyclictest1-21systemd07:35:171
2875199178,7cyclictest28775-21awk11:05:181
2875199178,7cyclictest17157-21perl08:05:181
2875199178,6cyclictest0-21swapper/108:45:291
2875199177,8cyclictest920-21kthreadcore08:20:211
2875199173,3cyclictest131rcu_preempt11:50:161
2875199172,2cyclictest131rcu_preempt09:51:231
2875199172,12cyclictest10470-21diskstats10:55:171
2875199172,11cyclictest4945-21apt-get09:00:111
2875199171,9cyclictest0-21swapper/111:55:181
2875199171,8cyclictest0-21swapper/109:15:401
2875199171,8cyclictest0-21swapper/109:15:391
2875199171,12cyclictest6083-21apt-get07:20:131
2875199171,12cyclictest6083-21apt-get07:20:131
2875199170,9cyclictest14312-21diskmemload11:19:191
2875199170,11cyclictest20961-21ssh10:25:121
2873999175,3cyclictest12-21ksoftirqd/007:50:210
2873999171,13cyclictest31144-21kthreadcore07:10:220
2873999171,13cyclictest31143-21sendmail08:20:010
28739991711,3cyclictest16723-21latency_hist08:05:010
28739991710,5cyclictest7436-21kthreadcore12:05:190
2877299169,5cyclictest0-21swapper/307:55:203
2877299168,6cyclictest0-21swapper/308:10:283
28772991610,2cyclictest0-21swapper/308:30:113
2875199169,5cyclictest19855-21cut08:40:201
2875199168,5cyclictest9043-21irqcore08:30:181
2875199162,10cyclictest12157-21apt-get08:00:131
28751991611,3cyclictest0-21swapper/107:50:021
2875199161,12cyclictest27720-21apt-get08:50:131
2875199161,11cyclictest32280-21ntp_192.168.11407:10:251
28751991610,4cyclictest10913-21mailstats07:55:271
2873999169,5cyclictest21874-21fschecks_count08:10:160
2873999168,5cyclictest3903-21fschecks_count08:25:150
2873999168,5cyclictest3903-21fschecks_count08:25:150
2873999162,11cyclictest27640-21df09:35:150
2873999162,11cyclictest24379-21kthreadcore08:45:200
2873999161,6cyclictest3549-21munin-run08:25:010
2873999161,11cyclictest6237-21grep07:20:170
2873999161,11cyclictest6237-21grep07:20:160
2876399150,8cyclictest28728-21cyclictest07:10:022
2875199156,7cyclictest15667-21sshd07:30:131
2875199156,6cyclictest13203-21kthreadcore07:25:211
2875199153,10cyclictest2546-21cpu07:50:181
2875199151,11cyclictest9754-21apt-get09:05:141
2873999154,9cyclictest5144-21fschecks_time09:00:150
2873999151,10cyclictest28325-21sed08:50:180
2875199131,9cyclictest28642-21latency_hist07:10:011
2873999121,8cyclictest28834-21grep07:10:020
2877299101,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional