You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-10-01 - 17:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Thu Oct 01, 2026 00:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2901099601,39cyclictest31804-21ssh23:25:131
29018995731,14cyclictest27-21ksoftirqd/221:40:142
2900299572,6cyclictest131rcu_preempt21:20:120
2900299511,5cyclictest131rcu_preempt23:20:130
2901099503,36cyclictest30083-21ssh22:30:111
2900299492,39cyclictest10174-21ssh22:55:120
2902699488,25cyclictest35966irq/58-eth000:30:133
29002994830,6cyclictest12-21ksoftirqd/021:10:130
2900299481,38cyclictest23227-21ssh00:35:110
2900299481,38cyclictest23227-21ssh00:35:110
2900299481,36cyclictest26437-21ssh22:10:120
29010994631,10cyclictest0-21swapper/123:00:131
29010994612,24cyclictest21-21ksoftirqd/120:25:131
2900299462,36cyclictest29830-21ssh00:00:110
2901899452,30cyclictest17305-21ssh23:35:132
2901099453,39cyclictest20810-21ssh23:55:101
29010994225,10cyclictest21-21ksoftirqd/100:30:111
2901099413,35cyclictest24559-21ssh22:45:111
2900299411,28cyclictest26303-21ssh23:40:120
2901899401,20cyclictest930-21snmpd22:00:132
2900299402,31cyclictest31799-21ssh23:25:120
2902699392,27cyclictest35966irq/58-eth023:20:143
2901099393,28cyclictest8357-21ssh23:30:111
2902699383,23cyclictest35966irq/58-eth000:10:093
2901899384,17cyclictest27-21ksoftirqd/223:40:132
2901899382,18cyclictest8633-21sh00:07:532
29010993825,7cyclictest14691-21diskmemload22:15:141
29010993814,3cyclictest131rcu_preempt22:40:141
2900299371,29cyclictest1150-21ssh22:50:140
2900299371,29cyclictest1150-21ssh22:50:130
2902699366,13cyclictest35966irq/58-eth022:05:123
2902699363,20cyclictest35966irq/58-eth022:20:113
2901099361,24cyclictest17291-21ssh23:35:121
2902699354,18cyclictest35966irq/58-eth021:20:133
2902699352,19cyclictest35966irq/58-eth023:25:123
2901099351,26cyclictest19758-21ssh00:15:131
2900299352,27cyclictest13654-21ssh21:45:120
2901899342,20cyclictest17438-21ssh22:05:132
29018993412,13cyclictest27-21ksoftirqd/222:10:122
2901099340,25cyclictest28145-21ssh23:05:121
2902699334,18cyclictest0-21swapper/322:10:123
2901899322,14cyclictest26622-21sh22:26:182
2900299321,22cyclictest15631-21ssh22:40:140
2902699311,22cyclictest35966irq/58-eth023:35:133
2901899312,23cyclictest2973-21ntp_192.168.11421:55:252
29010993117,8cyclictest21-21ksoftirqd/121:47:251
29010993115,10cyclictest0-21swapper/120:55:121
29010993115,10cyclictest0-21swapper/120:55:121
2900299312,6cyclictest131rcu_preempt19:10:120
2902699304,14cyclictest35966irq/58-eth022:40:143
2901099302,18cyclictest6572-21ssh22:35:121
2901899296,8cyclictest0-21swapper/221:10:122
2901099292,19cyclictest21-21ksoftirqd/121:55:151
2901099291,18cyclictest9948-21sh21:25:141
2901099291,18cyclictest9948-21sh21:25:141
2900299295,6cyclictest131rcu_preempt19:20:130
2900299295,6cyclictest131rcu_preempt19:20:130
29026992819,4cyclictest0-21swapper/319:25:113
2901899284,17cyclictest27346irq/40-dwc2_hso21:05:132
2901899283,20cyclictest0-21swapper/221:49:252
2901899282,21cyclictest9834-21ssh23:48:072
2900299282,22cyclictest5294-21ssh00:25:130
2900299282,20cyclictest8378-21/usr/sbin/munin22:00:250
2902699274,13cyclictest35966irq/58-eth021:50:113
2902699273,14cyclictest35966irq/58-eth019:40:143
2901899274,13cyclictest27346irq/40-dwc2_hso20:15:122
2901899272,4cyclictest131rcu_preempt23:55:112
2901099272,20cyclictest8331-21ntp_states21:40:251
29010992717,6cyclictest21-21ksoftirqd/100:35:121
29010992717,6cyclictest21-21ksoftirqd/100:35:121
29010992714,6cyclictest0-21swapper/122:05:141
2902699263,17cyclictest0-21swapper/323:57:543
2901899263,11cyclictest30098-21ssh22:30:122
2901899262,14cyclictest27346irq/40-dwc2_hso19:25:122
2900299265,15cyclictest12-21ksoftirqd/000:20:110
2900299262,19cyclictest11936-21ssh21:43:250
2900299261,19cyclictest14268-21ssh00:30:130
2901899255,14cyclictest27346irq/40-dwc2_hso19:45:132
2901899255,14cyclictest27346irq/40-dwc2_hso00:30:122
2901899255,14cyclictest0-21swapper/222:55:122
2901899254,13cyclictest27346irq/40-dwc2_hso00:20:112
2901899252,16cyclictest27346irq/40-dwc2_hso00:00:112
29018992513,9cyclictest0-21swapper/220:40:162
2901899251,15cyclictest11815-21ssh23:50:132
2901099253,17cyclictest21-21ksoftirqd/123:10:121
2901099253,15cyclictest664-21ntpd19:10:131
2902699243,7cyclictest0-21swapper/322:00:253
29026992419,3cyclictest35966irq/58-eth022:30:113
2901899244,14cyclictest27346irq/40-dwc2_hso21:20:132
2901899244,12cyclictest27346irq/40-dwc2_hso00:10:082
2901899242,18cyclictest18987-21ssh21:30:142
29018992417,5cyclictest4225-21ssh22:15:192
29018992416,4cyclictest8145-21ssh23:10:232
2901899241,20cyclictest27346irq/40-dwc2_hso23:20:132
2900299249,11cyclictest14691-21diskmemload21:50:120
2900299242,16cyclictest844-21munin-node19:25:130
29002992412,5cyclictest536-21usb-storage23:50:130
2902699236,11cyclictest0-21swapper/322:55:123
2901899238,7cyclictest8926-21ntpq00:25:252
2901899232,6cyclictest0-21swapper/222:38:252
2901899232,6cyclictest0-21swapper/221:50:132
29018992312,9cyclictest0-21swapper/220:35:182
29018992312,4cyclictest0-21swapper/220:50:162
29018992311,9cyclictest3371-21ssh21:39:052
2901899231,13cyclictest19212-21ssh23:00:142
2901099234,15cyclictest0-21swapper/122:55:101
2901099233,14cyclictest26425-21apt-get22:10:111
2901099231,19cyclictest25283-21ls20:10:231
29010992311,6cyclictest23461-21kthreadcore21:50:191
2901099231,11cyclictest19967-21cut20:40:201
2902699229,7cyclictest35966irq/58-eth000:35:123
2902699229,7cyclictest35966irq/58-eth000:35:123
2902699223,13cyclictest0-21swapper/323:48:073
2902699223,12cyclictest0-21swapper/323:40:133
2901899225,6cyclictest7137-21kthreadcore19:20:192
2901899225,6cyclictest7137-21kthreadcore19:20:192
2901899223,7cyclictest15637-21ssh22:40:142
2901899222,6cyclictest0-21swapper/219:40:232
2901899222,17cyclictest832-21kthreadcore20:20:202
2901899222,16cyclictest0-21swapper/223:09:322
2901899222,12cyclictest20807-21kthreadcore00:15:192
29018992212,8cyclictest0-21swapper/219:35:142
2901899221,17cyclictest553-21latency_hist21:20:022
2901899221,17cyclictest4283-21if_eth120:25:172
2901899221,16cyclictest8796-21apt-get20:30:132
2901899221,16cyclictest8796-21apt-get20:30:132
2901899221,13cyclictest12634-21ssh23:30:292
2901899221,11cyclictest7751-21fschecks_time19:55:152
2901899221,11cyclictest25653-21kthreadcore22:45:192
2901899221,11cyclictest1751-21kthreadcore23:25:212
2901899221,11cyclictest13796-21sort23:15:142
29018992210,9cyclictest19847-21unixbench_singl19:30:282
2901899220,19cyclictest9947-21ssh21:25:142
2901899220,19cyclictest9947-21ssh21:25:142
2901099228,10cyclictest0-21swapper/120:05:131
2901099226,12cyclictest10128-21ssh00:25:311
2901099222,16cyclictest27912-21ssh21:15:271
2901099222,14cyclictest24171-21ntp_192.168.11419:35:231
2901099222,14cyclictest12203-21ntp_192.168.11420:30:241
2901099222,14cyclictest12203-21ntp_192.168.11420:30:241
2901099221,17cyclictest31539-21sshd20:50:251
2901099221,14cyclictest26502-21cut23:40:151
2900299227,8cyclictest12-21ksoftirqd/022:45:130
2900299222,17cyclictest20826-21ssh23:55:120
2900299221,9cyclictest22631-21ntpq21:30:240
2900299221,17cyclictest3969-21find20:25:120
2900299221,15cyclictest25500-21ls19:40:020
2902699219,9cyclictest0-21swapper/300:07:043
2902699213,14cyclictest35966irq/58-eth021:10:043
2901899212,12cyclictest1312-21fschecks_count22:50:152
2901899212,12cyclictest1312-21fschecks_count22:50:152
2901899211,9cyclictest24137-21kthreadcore20:45:202
2901899211,9cyclictest23948-21kernelversion00:35:172
2901899211,9cyclictest23948-21kernelversion00:35:172
2901899211,17cyclictest6610-21grep19:50:282
2901899211,17cyclictest14705-21kthreadcore20:00:202
2901899211,12cyclictest5389-21ls21:00:142
2901899211,11cyclictest22527-21sed20:10:192
2901899211,11cyclictest21079-21grep20:05:282
29018992110,8cyclictest0-21swapper/222:20:122
2901099219,9cyclictest6107-21cron19:20:011
2901099213,10cyclictest0-21swapper/121:35:141
2901099212,16cyclictest22729-21apt-get23:20:131
2901099211,16cyclictest8807-21kthreadcore19:20:211
2901099211,16cyclictest8807-21kthreadcore19:20:211
2900299211,8cyclictest18187-21apt20:40:000
29002992113,6cyclictest0-21swapper/020:05:260
2900299211,16cyclictest6845-21ls22:35:150
2900299211,15cyclictest14691-21diskmemload21:35:150
2901899202,11cyclictest0-21swapper/219:15:152
2901899201,9cyclictest650-21grep20:55:162
2901899201,9cyclictest650-21grep20:55:162
2901899201,10cyclictest32368-21kthreadcore19:10:212
2901099209,9cyclictest0-21swapper/121:32:061
2901099208,9cyclictest11803-21apt-get23:50:121
2901099206,9cyclictest201rcuc/120:45:121
2901099203,14cyclictest4645-21/usr/sbin/munin22:50:221
2901099203,14cyclictest4645-21/usr/sbin/munin22:50:221
2901099203,12cyclictest14695-21apt-get21:10:021
2901099203,10cyclictest201rcuc/120:35:111
2901099202,9cyclictest131rcu_preempt22:21:561
2901099202,2cyclictest131rcu_preempt20:00:261
2901099202,2cyclictest131rcu_preempt00:24:001
29010992013,4cyclictest0-21swapper/121:20:061
2901099201,10cyclictest2807-21/usr/sbin/munin23:45:171
2901099200,18cyclictest24539-21ls22:25:201
2901099200,14cyclictest23871-21ls21:15:031
2900299209,8cyclictest17489-21awk22:05:140
2900299203,8cyclictest12-21ksoftirqd/021:25:190
2900299203,8cyclictest12-21ksoftirqd/021:25:190
2900299202,14cyclictest844-21munin-node19:15:150
29002992011,6cyclictest30143-21cron19:45:010
2900299201,16cyclictest15559-21latency22:20:210
2900299201,15cyclictest8377-21ssh23:30:120
29002992010,7cyclictest12-21ksoftirqd/023:15:130
2902699199,7cyclictest0-21swapper/323:00:233
2902699199,7cyclictest0-21swapper/300:18:233
2902699196,10cyclictest0-21swapper/322:35:203
2902699193,11cyclictest0-21swapper/322:26:253
2902699193,11cyclictest0-21swapper/321:56:103
2902699192,11cyclictest0-21swapper/300:27:573
29026991910,6cyclictest0-21swapper/323:15:173
29026991910,6cyclictest0-21swapper/323:09:023
29026991910,6cyclictest0-21swapper/322:52:273
29026991910,6cyclictest0-21swapper/322:52:263
29026991910,6cyclictest0-21swapper/321:36:593
2901099199,7cyclictest15129-21cat00:10:271
2901099192,2cyclictest131rcu_preempt19:55:451
2901099191,9cyclictest131rcu_preempt00:09:171
29010991914,3cyclictest0-21swapper/120:15:251
29010991912,4cyclictest9406-21kthreadcore22:00:181
29010991911,5cyclictest30286-21irqcore00:00:171
2901099191,15cyclictest2757-21find19:50:121
2900299196,3cyclictest14691-21diskmemload22:18:460
2900299193,13cyclictest17268-21find23:35:110
29002991913,4cyclictest0-21swapper/021:15:250
29002991911,6cyclictest16948-21sshd20:03:240
2900299191,15cyclictest14691-21diskmemload23:05:260
2902699189,6cyclictest0-21swapper/323:50:303
2902699189,6cyclictest0-21swapper/322:48:533
2902699189,6cyclictest0-21swapper/321:19:073
2902699185,10cyclictest35966irq/58-eth022:15:133
2902699182,9cyclictest35966irq/58-eth019:20:133
2902699182,9cyclictest35966irq/58-eth019:20:123
2902699182,10cyclictest0-21swapper/320:50:133
2902699182,10cyclictest0-21swapper/319:55:123
2902699181,9cyclictest35966irq/58-eth021:40:143
29026991811,4cyclictest0-21swapper/323:10:213
2902699181,12cyclictest0-21swapper/300:00:163
29026991810,6cyclictest0-21swapper/323:32:283
29026991810,6cyclictest0-21swapper/321:29:393
29026991810,6cyclictest0-21swapper/321:29:393
2901099189,6cyclictest31263-21kthreadcore19:45:181
2901099182,13cyclictest6004-21kthreadcore21:00:201
2901099181,14cyclictest13723-21apt-get23:15:121
2901099181,14cyclictest11171-21apt-get19:25:111
2901099180,3cyclictest131rcu_preempt19:40:281
2900299183,8cyclictest14691-21diskmemload23:48:020
2900299183,7cyclictest0-21swapper/022:26:180
2900299182,11cyclictest0-21swapper/020:40:140
2900299181,5cyclictest0-21swapper/020:10:190
29002991811,5cyclictest12926-21ssh23:14:320
2900299181,14cyclictest3189-21/usr/sbin/munin21:55:280
2900299181,13cyclictest19325-21diskstats23:00:160
29002991810,3cyclictest15077-21usage_Anybus00:10:260
2902699179,3cyclictest0-21swapper/300:24:403
2902699172,9cyclictest0-21swapper/319:15:163
2902699172,8cyclictest0-21swapper/321:45:123
2902699172,11cyclictest0-21swapper/321:30:133
29026991711,4cyclictest0-21swapper/319:33:223
2902699171,10cyclictest0-21swapper/320:40:143
2901099171,8cyclictest1713-21kthreadcore20:20:211
29010991711,4cyclictest0-21swapper/119:10:021
2901099171,13cyclictest20545-21idleruntime-cro19:35:011
2900299178,6cyclictest1290-21ntpq22:30:250
2900299174,3cyclictest131rcu_preempt20:20:220
2900299173,8cyclictest0-21swapper/000:07:530
2900299172,10cyclictest0-21swapper/019:45:130
2900299171,13cyclictest5019-21apt-get21:00:050
2900299171,12cyclictest2183-21kthreadcore20:55:190
2900299171,12cyclictest2183-21kthreadcore20:55:190
2900299171,11cyclictest12651-21tr21:05:220
2900299171,10cyclictest0-21swapper/020:15:120
29002991710,5cyclictest21712-21kthreadcore00:15:210
2902699161,8cyclictest35966irq/58-eth020:15:123
2902699161,12cyclictest0-21swapper/320:55:273
2902699161,12cyclictest0-21swapper/320:55:273
2902699161,12cyclictest0-21swapper/320:45:123
2902699161,12cyclictest0-21swapper/320:25:013
2901899161,11cyclictest29040-21latency_hist19:10:022
2900299168,6cyclictest26755-21grep20:45:280
2900299162,10cyclictest0-21swapper/020:50:140
2900299162,10cyclictest0-21swapper/020:30:140
2900299162,10cyclictest0-21swapper/020:30:140
2900299161,12cyclictest17167-21tr19:30:200
29002991610,4cyclictest0-21swapper/019:55:160
29002991610,3cyclictest4037-21cut19:50:190
2902699154,7cyclictest0-21swapper/321:00:143
2902699151,10cyclictest0-21swapper/319:45:203
2902699149,3cyclictest0-21swapper/319:54:483
2902699143,5cyclictest0-21swapper/319:35:123
2902699142,10cyclictest0-21swapper/321:10:023
2902699141,7cyclictest0-21swapper/320:05:163
2900299141,10cyclictest29039-21cat19:10:020
2902699138,3cyclictest0-21swapper/320:30:243
2902699138,3cyclictest0-21swapper/320:30:243
2902699138,3cyclictest0-21swapper/320:25:263
2902699138,3cyclictest0-21swapper/319:12:023
2902699133,3cyclictest0-21swapper/320:10:143
2902699131,5cyclictest0-21swapper/320:00:113
2902699122,8cyclictest0-21swapper/320:35:233
2902699111,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional