You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 07:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Mar 14, 2026 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2717999521,44cyclictest20829-21ssh21:50:131
2719599421,29cyclictest28366irq/58-eth020:00:113
2717999412,5cyclictest131rcu_preempt20:15:131
2717999411,28cyclictest6460-21ssh22:00:121
2719599396,18cyclictest28366irq/58-eth023:20:123
2719599396,18cyclictest28366irq/58-eth023:20:123
27187993821,4cyclictest131rcu_preempt22:45:152
2719599372,21cyclictest28366irq/58-eth022:05:113
2719599372,21cyclictest28366irq/58-eth022:05:113
27187993721,12cyclictest0-21swapper/219:45:112
27179993727,7cyclictest13058-21diskmemload21:45:121
2717999372,27cyclictest19251-21rm22:25:141
27179993414,10cyclictest27157-21cyclictest19:55:141
2718799331,22cyclictest26212-21ssh21:35:122
2717999332,25cyclictest16286-21sh00:28:561
2717099325,23cyclictest5037-21ssh00:05:110
2717099322,25cyclictest3112-21ssh23:10:120
2718799312,17cyclictest10439-21sh23:50:132
2717999311,22cyclictest3106-21ssh23:10:111
2719599301,17cyclictest28366irq/58-eth022:10:133
2718799303,24cyclictest31519-21ssh21:20:122
27187993023,4cyclictest924-21snmpd20:40:142
2719599293,15cyclictest28366irq/58-eth023:55:123
2719599292,17cyclictest28366irq/58-eth022:15:143
27187992924,3cyclictest22346irq/40-dwc2_hso23:25:132
2717099291,25cyclictest22971-21ssh00:15:110
2718799284,18cyclictest22346irq/40-dwc2_hso19:20:112
2718799283,20cyclictest10175-21ssh22:20:132
2718799283,20cyclictest10175-21ssh22:20:132
2718799282,18cyclictest8152-21ssh21:25:132
2718799282,15cyclictest27-21ksoftirqd/200:15:112
2717099284,22cyclictest31502-21ssh21:20:110
2717099281,24cyclictest20808-21ssh21:50:110
2719599278,13cyclictest28366irq/58-eth000:00:133
2719599273,16cyclictest0-21swapper/300:30:213
2719599273,14cyclictest28366irq/58-eth000:20:163
2717999272,19cyclictest19471-21ssh23:55:131
2717999272,18cyclictest21085-21memory00:30:211
27179992715,9cyclictest7071-21cron20:30:011
2717099272,22cyclictest10435-21ssh23:50:120
2717099272,21cyclictest22890-21ssh22:45:160
2717099272,20cyclictest13456-21ssh21:10:130
2719599261,18cyclictest28366irq/58-eth023:45:113
2718799264,13cyclictest22346irq/40-dwc2_hso20:05:132
2718799263,14cyclictest22346irq/40-dwc2_hso21:10:122
2718799261,22cyclictest9749-21ls22:20:022
27187992611,6cyclictest26112-21cron23:05:012
2717999261,20cyclictest2799-21ssh21:40:131
2717099261,21cyclictest8503-21ssh22:55:130
2717099261,21cyclictest8503-21ssh22:55:130
2717099261,21cyclictest17211-21ssh21:30:130
2717099261,21cyclictest17211-21ssh21:30:120
2718799252,17cyclictest22346irq/40-dwc2_hso23:40:132
2718799252,17cyclictest22346irq/40-dwc2_hso23:40:132
2718799252,16cyclictest22346irq/40-dwc2_hso21:00:122
27187992518,5cyclictest261rcuc/221:50:112
2718799251,7cyclictest1073166ssh21:44:122
2718799251,6cyclictest1596166ssh00:28:322
2718799251,11cyclictest3130-21diskmemload23:10:132
27179992513,9cyclictest31501-21apt-get21:20:121
2717999251,20cyclictest32068-21ssh00:20:161
2717999251,17cyclictest12089-21ssh23:15:111
2717099253,18cyclictest29824-21ssh21:55:120
2717099252,19cyclictest31884-21ssh22:50:130
2717099251,20cyclictest6773-21ssh23:30:120
2719599249,11cyclictest28366irq/58-eth021:15:123
2719599244,11cyclictest28366irq/58-eth021:50:123
2718799244,13cyclictest22346irq/40-dwc2_hso19:30:122
2718799242,19cyclictest0-21swapper/221:30:122
2718799242,19cyclictest0-21swapper/221:30:122
2718799242,19cyclictest0-21swapper/220:25:132
2718799241,6cyclictest3149866rm22:13:042
2718799241,14cyclictest127866ssh23:45:032
2717099247,13cyclictest4815-21ssh22:35:120
2717099246,9cyclictest16885-21grep21:30:010
2717099241,18cyclictest15475-21apt-get22:05:120
2717099241,18cyclictest15475-21apt-get22:05:110
27195992313,6cyclictest0-21swapper/300:15:123
2718799234,14cyclictest22346irq/40-dwc2_hso00:05:122
2718799232,4cyclictest8704-21kthreadcore20:30:182
27187992311,9cyclictest10453-21cron20:00:012
27187992311,5cyclictest2547-21ssh23:09:532
27187992310,10cyclictest20023-21kthreadcore00:30:192
2718799230,7cyclictest2044666ssh00:12:242
2718799230,6cyclictest2562666ssh22:27:222
2717999232,17cyclictest22438-21ssh21:15:131
27179992311,9cyclictest28481-21perl22:30:151
2717999231,19cyclictest10406-21find23:50:111
2717999230,11cyclictest131rcu_preempt20:00:111
2717099234,3cyclictest131rcu_preempt23:45:300
27170992314,7cyclictest12-21ksoftirqd/019:30:110
2719599223,13cyclictest0-21swapper/300:28:563
2719599221,16cyclictest28366irq/58-eth022:00:123
2718799223,9cyclictest13058-21diskmemload23:30:122
2718799223,10cyclictest31178-21latency_hist20:55:012
2718799222,8cyclictest4839-21ssh22:35:132
2718799222,7cyclictest0-21swapper/219:10:122
2718799222,15cyclictest24846-21ntp_192.168.11423:20:232
2718799222,15cyclictest24846-21ntp_192.168.11423:20:232
2718799222,11cyclictest23229-21ssh23:55:262
2718799221,7cyclictest2159566ssh23:36:412
2718799221,6cyclictest273766ssh22:32:592
2718799221,6cyclictest1942966ssh23:18:222
2718799221,6cyclictest1935166ssh22:41:162
27187992211,8cyclictest3382-21proc_pri00:20:282
2718799221,13cyclictest1415066ssh22:56:252
2718799221,13cyclictest1415066ssh22:56:252
2718799221,13cyclictest13056-21sh21:10:012
2718799221,12cyclictest19140-21ssh22:05:242
2718799221,12cyclictest19140-21ssh22:05:242
2718799221,12cyclictest1809-21kthreadcore22:50:212
27187992210,9cyclictest25189-21latency_hist20:15:012
2717999224,5cyclictest131rcu_preempt00:35:111
2717999222,16cyclictest9312-21apt-get19:25:121
2717099222,9cyclictest0-21swapper/021:35:120
2717099221,4cyclictest19226-21apt-get22:25:130
27170992214,5cyclictest12-21ksoftirqd/020:05:130
2717099221,18cyclictest30083-21ls00:35:220
2717099221,17cyclictest13897-21ssh22:40:120
2717099221,14cyclictest24519-21ssh22:10:120
2719599219,7cyclictest28366irq/58-eth021:45:133
27195992111,4cyclictest0-21swapper/300:10:223
2718799211,8cyclictest1804466ssh21:47:082
2718799211,7cyclictest2992366ssh21:55:132
2718799211,6cyclictest3027266ssh21:19:012
2718799211,16cyclictest28624-21df00:00:142
2718799211,16cyclictest19153-21cut19:35:152
2718799211,11cyclictest4292-21idleruntime-cro19:20:012
2718799211,11cyclictest25949-21tr20:15:192
2718799211,11cyclictest22379-21kernelversion20:45:172
2718799211,10cyclictest5831-21grep19:55:022
27187992110,8cyclictest12929-21ssh22:02:262
2717999216,12cyclictest32617-21grep00:00:291
2717999213,15cyclictest6390-21ssh23:30:011
2717999213,13cyclictest14009-21ssh00:10:131
27179992112,7cyclictest1-21systemd23:42:091
27179992112,7cyclictest1-21systemd23:42:091
27179992112,6cyclictest16690-21cron20:40:011
27179992111,8cyclictest9675-21cron22:20:011
27179992111,7cyclictest26275-21cron20:50:011
2717999211,17cyclictest13705-21sh22:40:021
2717999210,17cyclictest25761-21ps19:40:201
2717099212,4cyclictest30319-21fschecks_time20:20:150
2717099212,10cyclictest0-21swapper/020:50:130
2717099211,8cyclictest14636-21cut00:10:180
2717099211,4cyclictest0-21swapper/023:57:390
2717099211,3cyclictest0-21swapper/023:44:120
2717099211,3cyclictest0-21swapper/023:44:120
2717099211,3cyclictest0-21swapper/023:27:530
2717099211,3cyclictest0-21swapper/021:47:380
2717099211,3cyclictest0-21swapper/000:20:300
2717099210,18cyclictest5991-21cut19:55:170
2719599209,6cyclictest0-21swapper/323:52:273
27195992011,6cyclictest28366irq/58-eth023:40:133
27195992011,6cyclictest28366irq/58-eth023:40:133
2719599201,12cyclictest0-21swapper/319:50:173
27195992010,8cyclictest0-21swapper/323:38:143
27195992010,4cyclictest0-21swapper/321:26:393
2718799203,8cyclictest24242-21cut19:40:172
2718799202,14cyclictest22346irq/40-dwc2_hso19:25:122
27187992015,3cyclictest924-21snmpd20:00:122
2718799201,11cyclictest27528-21grep00:35:182
2718799201,10cyclictest13125-21kthreadcore20:35:182
2717999209,3cyclictest131rcu_preempt21:10:011
2717999204,12cyclictest0-21swapper/119:45:121
2717999202,14cyclictest15503-21ssh22:05:131
2717999202,14cyclictest15503-21ssh22:05:131
2717999202,13cyclictest20089-21apt-get20:10:031
2717999202,13cyclictest0-21swapper/100:15:211
27179992013,4cyclictest0-21swapper/123:45:041
27179992012,5cyclictest10632-21kthreadcore21:25:211
2717999201,15cyclictest7385-21timerandwakeup21:00:271
2717999201,10cyclictest0-21swapper/122:50:211
27179992010,7cyclictest20197-21cat23:35:301
2717099208,9cyclictest12-21ksoftirqd/022:20:140
2717099208,9cyclictest12-21ksoftirqd/022:20:140
2717099202,7cyclictest0-21swapper/020:35:110
2717099201,6cyclictest26508-21apt-get23:05:130
2717099201,4cyclictest0-21swapper/023:22:090
2717099201,4cyclictest0-21swapper/023:22:090
2717099201,3cyclictest0-21swapper/023:04:260
2717099201,3cyclictest0-21swapper/022:00:380
2717099201,16cyclictest28520-21apt-get00:00:120
2717099201,10cyclictest0-21swapper/023:15:540
2719599199,6cyclictest28366irq/58-eth023:30:123
2719599198,8cyclictest0-21swapper/322:34:163
2719599192,12cyclictest28366irq/58-eth023:05:133
2719599192,10cyclictest0-21swapper/319:15:133
2719599191,12cyclictest0-21swapper/322:20:293
2719599191,12cyclictest0-21swapper/322:20:293
2719599191,11cyclictest0-21swapper/321:45:023
27195991910,6cyclictest28366irq/58-eth000:35:233
27195991910,6cyclictest0-21swapper/321:56:093
2718799191,9cyclictest31496-21cut20:55:162
2718799191,9cyclictest30586-21cut20:20:162
2718799191,15cyclictest27245-21latency_hist19:10:012
2717999199,7cyclictest9096-21kthreadcore23:30:191
2717999197,9cyclictest31622-21tr21:55:191
2717999193,3cyclictest131rcu_preempt22:40:161
2717999192,14cyclictest22152-21latency_hist21:15:021
2717999191,8cyclictest131rcu_preempt23:00:111
2717999191,14cyclictest20915-21ssh21:30:251
2717999191,14cyclictest20915-21ssh21:30:251
2717999191,13cyclictest31611-21irqcore20:55:171
2717999190,14cyclictest14110-21ls22:20:281
2717999190,14cyclictest14110-21ls22:20:281
2717999190,11cyclictest29949-21ls23:25:011
2717999190,11cyclictest29949-21ls23:25:011
2717099196,11cyclictest0-21swapper/020:10:050
2717099192,3cyclictest131rcu_preempt00:25:230
2717099191,3cyclictest0-21swapper/022:17:420
2717099191,2cyclictest0-21swapper/000:31:340
2717099191,12cyclictest24016-21uniq19:40:150
2719599189,6cyclictest0-21swapper/323:25:543
2719599189,4cyclictest0-21swapper/321:31:553
2719599189,4cyclictest0-21swapper/321:31:553
2719599182,12cyclictest0-21swapper/321:10:153
2719599182,11cyclictest28366irq/58-eth019:45:113
2719599182,10cyclictest0-21swapper/320:15:043
2719599182,10cyclictest0-21swapper/319:55:153
27195991812,4cyclictest0-21swapper/323:10:123
27195991811,5cyclictest0-21swapper/321:20:213
2719599181,12cyclictest0-21swapper/323:15:273
2719599181,11cyclictest0-21swapper/320:45:273
27195991810,6cyclictest0-21swapper/322:25:163
2717999187,8cyclictest0-21swapper/123:05:141
2717999187,7cyclictest29868-21sh21:35:251
2717999185,4cyclictest131rcu_preempt22:10:151
2717999182,12cyclictest5062-21ssh00:05:131
2717999181,8cyclictest25111-21kthreadcore22:45:241
2717999181,8cyclictest1517-21sed20:20:271
2717999181,7cyclictest23728-21sort19:40:011
27179991812,4cyclictest0-21swapper/120:40:281
2717099188,8cyclictest12-21ksoftirqd/022:34:050
2717099184,2cyclictest131rcu_preempt23:38:150
2717099181,3cyclictest0-21swapper/019:15:170
2717099181,13cyclictest2883-21df21:40:140
2719599179,6cyclictest0-21swapper/322:54:293
2719599179,6cyclictest0-21swapper/322:46:043
2719599179,5cyclictest28366irq/58-eth022:55:133
2719599179,5cyclictest28366irq/58-eth022:55:133
2719599179,3cyclictest0-21swapper/323:00:443
27195991711,4cyclictest0-21swapper/322:35:183
2719599171,11cyclictest0-21swapper/319:40:213
2719599171,10cyclictest0-21swapper/321:35:303
2717999179,5cyclictest0-21swapper/119:10:151
2717999177,8cyclictest26510-21apt-get20:50:121
2717999172,12cyclictest20089-21apt-get20:10:011
2717999172,12cyclictest14751-21irqrtprio19:30:171
2717999171,4cyclictest0-21swapper/122:55:231
2717999171,4cyclictest0-21swapper/122:55:231
2717999171,12cyclictest2166-21kthreadcore19:15:201
2717999171,12cyclictest12062-21grep20:35:011
2717099179,5cyclictest24464-21kthreadcore20:45:200
2717099178,7cyclictest19052-21df19:35:140
2717099178,7cyclictest0-21swapper/020:25:120
2717099172,10cyclictest0-21swapper/021:05:110
2717099172,10cyclictest0-21swapper/021:00:140
2717099171,9cyclictest17100-21cut20:40:160
2717099171,13cyclictest924-21snmpd20:30:180
2717099171,13cyclictest22536-21diskstats21:15:140
2719599168,6cyclictest0-21swapper/300:05:073
2719599161,10cyclictest0-21swapper/320:50:143
27195991610,4cyclictest0-21swapper/322:40:133
2717999161,11cyclictest4666-21fschecks_count19:20:131
2717999161,10cyclictest0-21swapper/119:50:151
2717099167,7cyclictest10984-21if_eth020:00:150
2717099163,11cyclictest12920-21mailstats19:25:270
2717099162,10cyclictest0-21swapper/020:55:130
2717099162,10cyclictest0-21swapper/019:10:130
2717099161,2cyclictest131rcu_preempt19:50:140
2717099161,12cyclictest6226-21cut19:20:180
2717099161,12cyclictest27220-21tr20:15:210
2717099161,11cyclictest29348-21kernelversion19:45:170
2719599151,12cyclictest0-21swapper/321:00:273
2719599151,12cyclictest0-21swapper/319:35:013
2719599151,11cyclictest0-21swapper/321:05:133
2719599151,11cyclictest0-21swapper/320:56:323
2719599151,10cyclictest0-21swapper/320:20:193
2717999151,10cyclictest27249-21cat19:10:011
2719599149,3cyclictest0-21swapper/320:05:133
2719599141,7cyclictest0-21swapper/320:10:233
2719599141,10cyclictest0-21swapper/319:25:143
2719599138,3cyclictest0-21swapper/320:40:173
2717099131,9cyclictest27233-21latency_hist19:10:010
2719599126,4cyclictest28366irq/58-eth019:10:183
2719599121,8cyclictest0-21swapper/320:25:183
2719599121,5cyclictest0-21swapper/319:35:163
2719599116,3cyclictest0-21swapper/319:20:153
2719599111,7cyclictest0-21swapper/320:35:143
2719599111,6cyclictest0-21swapper/320:30:153
2719599101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional