You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 14:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sun May 17, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26603996520,15cyclictest31456-21ssh12:20:132
2658099643,10cyclictest131rcu_preempt09:00:140
2658099595,10cyclictest131rcu_preempt07:55:130
2660399573,19cyclictest25386-21sh09:35:132
2658099526,7cyclictest131rcu_preempt08:05:130
2658099522,39cyclictest12423-21ssh09:10:120
2662099514,26cyclictest35966irq/58-eth011:00:123
26603995038,9cyclictest0-21swapper/207:35:112
2662099492,33cyclictest35966irq/58-eth011:30:143
2662099483,28cyclictest35966irq/58-eth010:50:133
2662099483,28cyclictest35966irq/58-eth010:50:133
2658099473,27cyclictest2524-21ssh11:10:150
2662099463,27cyclictest35966irq/58-eth012:15:133
2659299461,39cyclictest13537-21apt-get07:30:111
2662099452,24cyclictest35966irq/58-eth009:05:133
2659299451,37cyclictest0-21swapper/111:05:131
2659299450,39cyclictest14753-21apt-get08:05:141
2662099447,20cyclictest35966irq/58-eth011:25:143
2662099442,24cyclictest35966irq/58-eth011:35:133
2659299441,38cyclictest31448-21apt-get12:20:141
2659299441,34cyclictest16458-21diskstats09:30:151
2659299431,39cyclictest26561-21cyclictest08:20:141
2659299431,37cyclictest17007-21apt-get12:30:121
2659299421,37cyclictest0-21swapper/110:35:121
2659299421,36cyclictest9612-21ssh10:20:141
2658099422,37cyclictest24236-21ssh11:40:130
2660399412,11cyclictest4575-21df_inode12:05:142
2660399412,11cyclictest4575-21df_inode12:05:142
2659299412,35cyclictest7392-21apt-get09:05:131
2659299412,34cyclictest0-21swapper/109:35:121
2659299411,30cyclictest26631-21ntp_states07:40:281
2659299411,25cyclictest30661-21ssh10:49:451
2659299410,25cyclictest26955-21ssh10:29:431
2658099415,26cyclictest12-21ksoftirqd/008:20:130
2662099402,21cyclictest35966irq/58-eth011:50:133
2659299402,33cyclictest2513-21apt-get11:10:151
2658099402,26cyclictest20121-21ssh09:50:140
2662099395,25cyclictest35966irq/58-eth012:05:133
2662099395,25cyclictest35966irq/58-eth012:05:133
2662099392,23cyclictest35966irq/58-eth011:10:153
26603993926,11cyclictest11945-21diskmemload12:10:132
2659299393,31cyclictest12412-21apt-get09:10:131
2659299391,33cyclictest8702-21apt-get07:25:131
2659299391,32cyclictest22422-21ssh12:15:121
2659299390,33cyclictest18321-21apt-get07:35:121
2658099395,10cyclictest131rcu_preempt11:15:140
2658099395,10cyclictest131rcu_preempt11:15:140
2658099391,30cyclictest5873-21ssh10:00:140
2658099391,28cyclictest29231-21ssh09:55:140
2659299382,32cyclictest0-21swapper/110:00:131
2659299382,31cyclictest5163-21apt-get07:55:131
2659299380,34cyclictest11053-21apt-get08:35:111
26580993814,18cyclictest5804-1kworker/u9:212:10:140
26620993714,15cyclictest35966irq/58-eth010:55:143
2659299372,31cyclictest7276-21apt-get09:25:121
2659299372,28cyclictest0-21swapper/108:45:131
2659299371,22cyclictest29426-21fschecks_time09:55:171
2659299371,21cyclictest11885-21if_eth111:15:181
2659299371,21cyclictest11885-21if_eth111:15:181
2659299370,27cyclictest11945-21diskmemload10:57:501
2659299370,23cyclictest13485-21ssh11:33:271
2659299362,31cyclictest813-21apt-get11:45:131
2659299362,31cyclictest0-21swapper/109:40:141
2659299362,31cyclictest0-21swapper/109:40:131
2659299361,31cyclictest2337-21memory10:50:221
2659299361,31cyclictest2337-21memory10:50:221
2659299361,30cyclictest0-21swapper/110:15:121
2659299361,29cyclictest11006-21apt-get09:45:131
2659299361,25cyclictest3093066ssh11:42:421
2659299361,22cyclictest19307-21ntp_states08:40:241
2659299361,19cyclictest21062-21sh11:36:431
2662099353,20cyclictest35966irq/58-eth011:40:133
2659299352,29cyclictest0-21swapper/112:35:121
2659299351,30cyclictest6536-21if_eth108:30:171
2659299351,30cyclictest27673-21apt-get10:30:131
2659299351,29cyclictest13308-21apt-get10:40:151
2659299350,32cyclictest9942-21apt-get08:00:161
26592993418,13cyclictest0-21swapper/111:25:131
2659299341,29cyclictest0-21swapper/110:05:111
2659299341,28cyclictest0-21swapper/112:05:121
2659299341,28cyclictest0-21swapper/112:05:111
2659299341,27cyclictest13416-21apt-get12:10:131
2659299340,24cyclictest2346-21ssh12:02:561
2662099332,17cyclictest35966irq/58-eth008:05:143
2662099331,16cyclictest35966irq/58-eth011:55:123
2659299331,29cyclictest16065-21ssh11:52:121
2659299331,25cyclictest0-21swapper/108:10:141
2659299331,23cyclictest11945-21diskmemload09:22:221
2659299330,29cyclictest309-21apt-get07:50:131
26580993316,12cyclictest12-21ksoftirqd/008:25:130
2660399329,12cyclictest27-21ksoftirqd/211:50:102
26592993216,13cyclictest0-21swapper/108:50:121
2659299321,28cyclictest3105-21irqrtprio09:00:201
2659299321,27cyclictest26875-21grep07:10:171
26580993222,6cyclictest12-21ksoftirqd/012:05:130
26580993222,6cyclictest12-21ksoftirqd/012:05:130
26580993220,4cyclictest536-21usb-storage09:20:110
2660399318,17cyclictest0-21swapper/211:35:122
2660399313,18cyclictest27346irq/40-dwc2_hso08:05:132
26592993116,9cyclictest26561-21cyclictest12:25:131
2659299311,27cyclictest26360-21ssh11:58:451
2659299311,26cyclictest28859-21users08:15:291
2658099318,18cyclictest12-21ksoftirqd/012:20:130
2658099314,24cyclictest4276-21ssh10:35:130
2662099303,15cyclictest35966irq/58-eth011:15:143
2662099303,15cyclictest35966irq/58-eth011:15:143
2662099302,13cyclictest35966irq/58-eth007:25:113
2660399303,18cyclictest27346irq/40-dwc2_hso08:40:112
2660399303,18cyclictest11945-21diskmemload11:05:132
2660399303,17cyclictest27346irq/40-dwc2_hso08:25:132
2660399302,11cyclictest3255-21ssh11:26:592
2659299302,25cyclictest802-21cut07:15:211
2659299301,19cyclictest24083-21memory11:20:231
2659299300,28cyclictest27224-21ssh09:16:341
2658099307,17cyclictest12-21ksoftirqd/008:00:150
2658099302,15cyclictest23930-21ssh10:10:140
2662099292,15cyclictest35966irq/58-eth011:45:133
2660399293,11cyclictest0-21swapper/211:43:002
2660399292,19cyclictest27346irq/40-dwc2_hso07:30:112
2660399292,11cyclictest10236-21ssh11:13:592
26592992918,6cyclictest0-21swapper/108:55:141
2659299291,23cyclictest16991-21ssh11:00:141
2659299290,26cyclictest27968-21apt-get07:45:141
2659299290,20cyclictest1510-21apt-get08:25:131
2658099293,15cyclictest13316-21ssh10:40:130
2662099282,19cyclictest0-21swapper/309:25:123
2662099282,16cyclictest35966irq/58-eth010:00:143
26620992813,10cyclictest35966irq/58-eth011:20:143
2660399283,17cyclictest27346irq/40-dwc2_hso09:10:122
2660399283,10cyclictest0-21swapper/212:36:002
2660399283,10cyclictest0-21swapper/211:55:592
2660399282,9cyclictest22436-21ssh12:15:132
2660399282,13cyclictest14713-21ssh09:45:252
2660399282,10cyclictest23807-21ssh11:02:592
26603992816,9cyclictest0-21swapper/211:30:142
2659299281,24cyclictest25757-21ssh09:51:291
2659299281,24cyclictest24331-21ssh10:10:201
2659299281,23cyclictest6675-21kthreadcore07:20:211
2658099284,18cyclictest510-21ssh10:15:120
2658099282,20cyclictest26662-21ssh11:21:590
2658099282,20cyclictest15204-21ssh11:35:130
2658099282,19cyclictest6068-21ssh11:30:070
26580992812,6cyclictest12-21ksoftirqd/010:55:140
2660399276,18cyclictest27-21ksoftirqd/210:50:112
2660399276,18cyclictest27-21ksoftirqd/210:50:112
2660399274,18cyclictest27346irq/40-dwc2_hso10:45:132
2660399273,18cyclictest27346irq/40-dwc2_hso09:40:142
2660399273,18cyclictest27346irq/40-dwc2_hso09:40:142
2660399273,14cyclictest5863-21ssh10:00:132
2658099272,19cyclictest25729-21ssh11:57:580
2660399264,15cyclictest27346irq/40-dwc2_hso10:20:142
2660399263,7cyclictest0-21swapper/210:40:152
2660399263,18cyclictest27346irq/40-dwc2_hso12:00:142
2660399263,15cyclictest27346irq/40-dwc2_hso07:20:122
26603992613,10cyclictest3640-21cron07:20:012
2658099264,17cyclictest151rcuc/009:30:110
2658099264,11cyclictest0-21swapper/007:25:130
26580992619,4cyclictest17030-21ssh12:30:130
2662099251,16cyclictest35966irq/58-eth012:10:143
26620992510,6cyclictest0-21swapper/308:25:133
2660399253,8cyclictest19290-21kworker/2:108:10:142
2660399252,18cyclictest21536-21ssh09:15:152
2660399251,14cyclictest27346irq/40-dwc2_hso09:20:112
2662099244,13cyclictest0-21swapper/312:30:143
2662099243,14cyclictest35966irq/58-eth012:20:133
2662099242,16cyclictest35966irq/58-eth010:35:133
2662099242,11cyclictest35966irq/58-eth009:50:143
2660399243,14cyclictest27346irq/40-dwc2_hso07:55:112
2660399242,18cyclictest27346irq/40-dwc2_hso09:30:132
2660399242,17cyclictest27346irq/40-dwc2_hso07:25:122
2660399242,14cyclictest27346irq/40-dwc2_hso08:20:132
2660399241,16cyclictest27346irq/40-dwc2_hso07:45:132
2660399241,11cyclictest93066snmpd10:32:512
2659299241,19cyclictest26601-21cat07:10:021
2658099243,3cyclictest131rcu_preempt11:25:170
2658099243,18cyclictest0-21swapper/010:05:130
26580992415,5cyclictest10623-21cron09:45:010
26580992415,5cyclictest10623-21cron09:45:010
2658099241,18cyclictest27700-21ssh10:30:130
2662099231,16cyclictest0-21swapper/312:35:243
2660399233,16cyclictest27346irq/40-dwc2_hso11:45:132
2660399232,18cyclictest24251-21ssh11:20:252
2660399231,7cyclictest17015-21ssh12:30:122
2658099239,3cyclictest0-21swapper/011:06:090
2658099232,16cyclictest27264-21ls10:30:010
2658099231,17cyclictest7998-21ssh12:25:120
26580992311,6cyclictest0-21swapper/011:45:120
2662099221,12cyclictest0-21swapper/309:30:133
2660399228,5cyclictest6939-21kthreadcore10:35:212
2660399228,11cyclictest24209-21wc10:10:182
2660399223,15cyclictest27346irq/40-dwc2_hso09:05:132
2660399223,12cyclictest27346irq/40-dwc2_hso08:35:112
2660399221,11cyclictest25191-21idleruntime-cro08:50:022
2658099229,9cyclictest26561-21cyclictest12:36:000
2658099224,11cyclictest0-21swapper/007:35:110
2658099222,3cyclictest131rcu_preempt08:15:260
2662099212,12cyclictest0-21swapper/309:55:133
26620992112,6cyclictest35966irq/58-eth009:40:143
26620992112,6cyclictest35966irq/58-eth009:40:143
2660399219,9cyclictest12380-21usage_Anybus12:25:302
2660399218,6cyclictest16429-21cron11:00:012
2660399218,10cyclictest28765-21http09:55:022
2660399217,11cyclictest16913-21kthreadcore10:05:192
2660399215,6cyclictest28687-21latency08:50:212
2660399211,8cyclictest6230-21/usr/sbin/munin08:30:222
2660399211,17cyclictest27631-21kthreadcore07:10:202
2660399211,15cyclictest27346irq/40-dwc2_hso11:15:142
2660399211,15cyclictest27346irq/40-dwc2_hso11:15:132
2660399211,12cyclictest25153-21kthreadcore07:40:232
2660399211,11cyclictest18742-21ssh10:25:162
2660399211,10cyclictest4130-21grep07:50:272
2660399211,10cyclictest32758-21kthreadcore08:55:222
2658099213,4cyclictest131rcu_preempt12:00:140
2658099212,15cyclictest31270-21ssh10:50:120
2658099212,15cyclictest31270-21ssh10:50:120
2658099212,14cyclictest29379-21ssh12:17:590
2658099212,13cyclictest11034-21ssh09:45:130
26580992111,7cyclictest614-21dbus-daemon09:19:380
2658099210,4cyclictest131rcu_preempt08:55:320
2658099210,18cyclictest25894-21ls10:45:240
2662099209,8cyclictest0-21swapper/310:22:293
2662099209,8cyclictest0-21swapper/309:14:203
2662099203,11cyclictest0-21swapper/311:05:033
2662099201,12cyclictest35966irq/58-eth008:50:123
2660399207,10cyclictest29568-21sh09:55:172
2660399201,9cyclictest3720-21kthreadcore09:00:212
2660399201,16cyclictest7261-21acpi09:25:112
2660399201,15cyclictest28899-21idleruntime-cro08:20:012
2660399201,10cyclictest2543-21kthreadcore10:15:212
2658099202,13cyclictest9707-21ssh11:50:100
2658099201,3cyclictest131rcu_preempt09:29:060
26580992012,5cyclictest12-21ksoftirqd/009:35:120
2658099201,10cyclictest131rcu_preempt11:03:380
2662099199,6cyclictest35966irq/58-eth010:25:143
2662099193,8cyclictest35966irq/58-eth007:20:123
2662099193,11cyclictest0-21swapper/312:02:593
2662099192,11cyclictest35966irq/58-eth010:10:143
26620991913,4cyclictest35966irq/58-eth010:30:053
26620991913,4cyclictest0-21swapper/308:45:133
26620991912,4cyclictest0-21swapper/312:25:253
26620991911,6cyclictest0-21swapper/310:40:153
2662099191,12cyclictest0-21swapper/308:05:023
26620991910,6cyclictest0-21swapper/309:35:323
2660399192,9cyclictest11597-21kthreadcore08:00:232
2658099197,3cyclictest131rcu_preempt08:45:270
2658099196,5cyclictest3254-21cat07:15:310
2658099194,11cyclictest0-21swapper/008:35:120
2658099192,11cyclictest0-21swapper/007:40:150
2658099191,3cyclictest131rcu_preempt07:50:190
2658099191,14cyclictest14473-21users10:20:320
2658099190,2cyclictest131rcu_preempt08:40:130
2662099189,6cyclictest35966irq/58-eth009:20:123
2662099188,8cyclictest0-21swapper/309:18:593
2662099182,10cyclictest0-21swapper/307:50:133
2662099182,10cyclictest0-21swapper/307:40:163
2662099182,10cyclictest0-21swapper/307:35:123
2658099184,11cyclictest27484-21kthreadcore07:10:200
2658099182,2cyclictest131rcu_preempt09:05:170
2658099182,13cyclictest12-21ksoftirqd/008:30:130
2658099181,2cyclictest131rcu_preempt07:30:130
2658099180,14cyclictest29206-21ls08:50:270
2662099179,6cyclictest0-21swapper/310:48:103
2662099179,6cyclictest0-21swapper/310:16:023
2662099179,6cyclictest0-21swapper/310:05:033
2662099178,4cyclictest0-21swapper/309:45:133
2662099173,8cyclictest0-21swapper/307:30:153
2662099172,10cyclictest0-21swapper/308:20:143
2662099172,10cyclictest0-21swapper/308:15:123
2662099171,9cyclictest0-21swapper/309:05:023
2662099171,13cyclictest0-21swapper/308:15:013
2660399171,8cyclictest26561-21cyclictest07:10:022
2658099171,13cyclictest7617-21timerandwakeup07:20:270
26580991710,5cyclictest0-21swapper/008:10:170
2662099162,9cyclictest0-21swapper/308:40:113
2658099164,6cyclictest131rcu_preempt07:45:120
2662099151,8cyclictest0-21swapper/308:00:023
2662099151,8cyclictest0-21swapper/307:10:293
2662099151,10cyclictest0-21swapper/308:30:203
2662099149,3cyclictest0-21swapper/307:49:103
2662099141,10cyclictest0-21swapper/307:15:303
2658099142,9cyclictest26606-21latency_hist07:10:020
2662099124,6cyclictest0-21swapper/308:35:173
2662099122,8cyclictest0-21swapper/308:55:163
266209991,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional