You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-13 - 00:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot6.osadl.org (updated Wed Nov 12, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
114121800,4sleep0631399cyclictest10:12:230
188521570,4sleep2632799cyclictest10:12:332
3026121450,3sleep0631399cyclictest08:57:320
1773911040,4ptp4l401ktimersoftd/308:11:133
174802740,4sleep0101ktimersoftd/010:30:380
177391720,1ptp4l391rcuc/313:41:553
633499629,22cyclictest10900-21sh12:05:143
631899620,30cyclictest6131-21/usr/sbin/munin10:52:231
326052620,1sleep10-21swapper/110:12:161
6334996112,27cyclictest26557-21sh10:40:073
6334995914,28cyclictest7813-21sh13:08:363
289162570,2sleep00-21swapper/011:50:150
6327995426,4cyclictest33-21ksoftirqd/213:02:052
6327995425,5cyclictest33-21ksoftirqd/212:37:262
6327995424,6cyclictest33-21ksoftirqd/213:02:222
632799536,7cyclictest33-21ksoftirqd/211:42:032
632799536,7cyclictest33-21ksoftirqd/211:42:032
6327995325,4cyclictest33-21ksoftirqd/212:32:252
6327995324,5cyclictest33-21ksoftirqd/211:17:212
6327995240,3cyclictest33-21ksoftirqd/212:07:342
6327995224,9cyclictest33-21ksoftirqd/211:02:212
6327995224,5cyclictest33-21ksoftirqd/213:12:482
632799512,7cyclictest33-21ksoftirqd/211:42:192
632799512,7cyclictest33-21ksoftirqd/211:42:192
6327995125,4cyclictest33-21ksoftirqd/208:37:182
6327995124,4cyclictest33-21ksoftirqd/213:29:292
6327995124,4cyclictest33-21ksoftirqd/210:47:212
6327995123,4cyclictest33-21ksoftirqd/213:37:022
6327995122,10cyclictest33-21ksoftirqd/212:48:232
42942510,2sleep00-21swapper/012:32:190
6327995039,3cyclictest33-21ksoftirqd/213:22:242
6327995024,5cyclictest33-21ksoftirqd/208:42:272
6327995024,4cyclictest33-21ksoftirqd/210:19:462
6327995023,6cyclictest33-21ksoftirqd/211:34:442
6327995022,9cyclictest33-21ksoftirqd/213:09:332
6327995022,6cyclictest33-21ksoftirqd/209:32:152
6327995022,5cyclictest33-21ksoftirqd/212:47:052
6327995021,5cyclictest33-21ksoftirqd/210:39:012
633499490,4cyclictest17708-21ssh10:31:023
6327994940,3cyclictest33-21ksoftirqd/212:27:052
632799492,8cyclictest33-21ksoftirqd/212:12:082
6327994923,4cyclictest33-21ksoftirqd/209:57:152
6327994923,4cyclictest33-21ksoftirqd/209:57:152
59612494,34sleep10-21swapper/108:08:461
6327994823,4cyclictest33-21ksoftirqd/211:49:312
6327994820,4cyclictest33-21ksoftirqd/211:54:002
632799481,10cyclictest15568-21ssh13:17:232
632799481,10cyclictest15568-21ssh13:17:232
632799479,3cyclictest33-21ksoftirqd/213:37:292
632799474,7cyclictest33-21ksoftirqd/212:17:182
6327994735,3cyclictest33-21ksoftirqd/209:12:252
6327994734,3cyclictest33-21ksoftirqd/211:57:462
6327994723,8cyclictest33-21ksoftirqd/208:17:012
6327994722,4cyclictest33-21ksoftirqd/211:00:292
6327994721,4cyclictest33-21ksoftirqd/211:13:412
6327994720,5cyclictest33-21ksoftirqd/210:29:302
632799466,7cyclictest33-21ksoftirqd/209:27:252
632799463,9cyclictest33-21ksoftirqd/210:52:172
6327994621,10cyclictest33-21ksoftirqd/209:42:132
6327994621,10cyclictest33-21ksoftirqd/209:02:172
6327994619,4cyclictest33-21ksoftirqd/211:24:092
6313994634,6cyclictest9-21ksoftirqd/010:22:150
60642464,29sleep20-21swapper/208:10:092
632799455,9cyclictest33-21ksoftirqd/210:32:172
632799455,7cyclictest33-21ksoftirqd/211:27:182
6327994538,4cyclictest33-21ksoftirqd/211:07:152
632799453,6cyclictest121rcu_preempt09:37:182
6327994529,3cyclictest33-21ksoftirqd/208:22:102
632799452,7cyclictest33-21ksoftirqd/210:02:242
632799452,7cyclictest33-21ksoftirqd/210:02:242
6327994521,4cyclictest33-21ksoftirqd/212:02:172
6327994519,4cyclictest33-21ksoftirqd/212:28:102
6327994519,4cyclictest33-21ksoftirqd/210:42:532
6327994519,4cyclictest33-21ksoftirqd/210:26:402
6327994518,4cyclictest33-21ksoftirqd/212:54:102
6327994517,4cyclictest33-21ksoftirqd/208:35:132
631899451,21cyclictest31866-21diskmemload12:09:011
633499440,2cyclictest31866-21diskmemload13:19:263
633499440,2cyclictest31866-21diskmemload13:19:263
631899442,24cyclictest32514-21latency_hist10:47:031
6334994312,18cyclictest1760-21runrttasks09:13:143
632799436,7cyclictest33-21ksoftirqd/209:47:152
632799436,3cyclictest33-21ksoftirqd/208:27:202
632799435,6cyclictest33-21ksoftirqd/208:12:142
632799433,7cyclictest33-21ksoftirqd/208:47:302
6327994333,3cyclictest33-21ksoftirqd/210:07:312
6327994319,6cyclictest33-21ksoftirqd/209:52:012
632799431,7cyclictest7720-21munin-run09:22:002
6327994317,10cyclictest33-21ksoftirqd/208:52:162
631899430,24cyclictest15589-21gltestperf09:37:171
6313994330,8cyclictest9-21ksoftirqd/011:17:170
633499420,4cyclictest22837-21sh11:09:433
632799426,6cyclictest33-21ksoftirqd/208:57:182
632799425,8cyclictest33-21ksoftirqd/209:07:242
631899421,21cyclictest0-21swapper/111:07:311
633499414,33cyclictest10533-21tr09:27:163
633499411,35cyclictest13738-21taskset13:15:353
631899411,25cyclictest20026-21mailstats10:32:251
631899410,26cyclictest8271-21awk11:27:341
632799404,7cyclictest33-21ksoftirqd/209:22:202
631899401,30cyclictest537-21apt-get13:02:181
631899401,24cyclictest25925-21/usr/sbin/munin11:12:271
631899400,24cyclictest23061-21sed12:52:051
633499393,30cyclictest24419-21apt-get13:27:173
631899392,31cyclictest10411-21apt-get09:27:141
631899392,29cyclictest28178-21apt-get12:57:211
631899391,31cyclictest30958-21apt-get09:02:121
631899390,36cyclictest17097-21ssh10:29:521
631899390,24cyclictest10004-21cat12:37:251
589723917,12sleep09-21ksoftirqd/008:07:570
177391390,1ptp4l401ktimersoftd/312:00:033
177391390,1ptp4l401ktimersoftd/311:30:443
177391390,1ptp4l401ktimersoftd/310:54:133
177391390,1ptp4l401ktimersoftd/310:42:543
177391390,1ptp4l401ktimersoftd/310:21:573
633499383,28cyclictest19090-21apt-get08:37:123
633499380,5cyclictest17808-21head12:12:203
631899382,30cyclictest22478-21apt-get09:52:141
631899382,29cyclictest5638-21apt-get09:17:101
631899380,24cyclictest5901-21hddtemp_smartct13:07:221
6313993832,3cyclictest9-21ksoftirqd/012:02:030
321672380,4sleep0101ktimersoftd/013:01:250
177391380,1ptp4l401ktimersoftd/312:40:423
177391380,1ptp4l401ktimersoftd/312:29:243
177391380,1ptp4l401ktimersoftd/312:18:013
177391380,1ptp4l401ktimersoftd/311:53:533
177391380,1ptp4l401ktimersoftd/311:39:193
177391380,1ptp4l401ktimersoftd/311:39:193
633499379,24cyclictest4531-21grep10:17:023
633499377,27cyclictest17374-21irqrtprio08:32:173
633499373,28cyclictest0-21swapper/309:47:113
633499372,2cyclictest121rcu_preempt12:08:523
633499372,29cyclictest0-21swapper/309:57:153
633499372,29cyclictest0-21swapper/309:57:153
633499372,27cyclictest25562-21apt-get11:47:163
633499370,5cyclictest6514-21irqrtprio09:17:163
631899373,30cyclictest27054-21grep12:22:181
631899370,23cyclictest25630-21kworker/u8:112:35:011
177391370,1ptp4l401ktimersoftd/312:47:253
177391370,1ptp4l401ktimersoftd/312:36:153
177391370,1ptp4l401ktimersoftd/311:44:133
177391370,1ptp4l401ktimersoftd/311:44:133
177391370,1ptp4l401ktimersoftd/311:22:113
177391370,1ptp4l401ktimersoftd/311:13:163
177391370,1ptp4l401ktimersoftd/310:58:323
631899361,31cyclictest8188-21unixbench_multi08:12:291
631899361,28cyclictest12775-21apt-get09:32:141
631899361,22cyclictest24754-21snmp_o5powerctl10:37:291
177391360,1ptp4l401ktimersoftd/311:21:543
177391360,1ptp4l401ktimersoftd/310:25:023
177391360,0ptp4l401ktimersoftd/312:25:293
177391360,0ptp4l401ktimersoftd/310:49:313
633499352,26cyclictest19674-21apt-get13:22:193
633499352,25cyclictest26208-21apt-get08:52:123
633499350,5cyclictest24131-21fschecks_count08:47:163
633499350,5cyclictest15327-21sed12:42:333
633499350,4cyclictest23009-21wc10:37:023
633499350,4cyclictest22267-21readlink09:52:013
633499350,3cyclictest21887-21grep08:42:153
631899353,26cyclictest13733-21apt-get12:42:161
631899352,26cyclictest22126-21apt-get12:17:181
631899352,26cyclictest21479-21apt-get08:42:111
6318993514,18cyclictest30739-21sed10:07:221
631899351,27cyclictest7551-21apt-get12:02:171
631899350,21cyclictest32607-21/usr/sbin/munin10:47:281
6313993529,4cyclictest9-21ksoftirqd/013:02:240
177391350,1ptp4l401ktimersoftd/309:38:313
177391350,0ptp4l401ktimersoftd/313:03:313
177391350,0ptp4l401ktimersoftd/312:53:083
633499342,24cyclictest28575-21apt-get08:57:143
6334993415,16cyclictest18918-21sh11:06:173
633499340,4cyclictest32077-21readlink10:12:013
633499340,4cyclictest31238-21ssh13:32:543
631899343,24cyclictest9576-21apt-get10:22:151
631899342,27cyclictest0-21swapper/109:42:121
631899342,25cyclictest3278-21apt-get09:12:151
6318993413,18cyclictest30279-21/usr/sbin/munin11:52:211
631899341,26cyclictest27439-21apt-get10:02:141
631899341,26cyclictest27439-21apt-get10:02:141
631899340,30cyclictest23416-21grep12:52:181
631899340,2cyclictest0-21swapper/113:27:301
6313993429,3cyclictest9-21ksoftirqd/012:42:050
6313993429,3cyclictest9-21ksoftirqd/010:42:020
6313993424,8cyclictest9-21ksoftirqd/011:07:250
317152340,4sleep0631399cyclictest12:27:170
6334993315,15cyclictest6775-21expr08:12:143
6334993315,15cyclictest28096-21/usr/sbin/munin12:57:273
633499330,4cyclictest11897-21sed08:21:593
631899333,26cyclictest26135-21if_enp1s011:47:211
631899333,26cyclictest0-21swapper/110:17:151
631899332,25cyclictest10128-21apt-get10:57:141
631899332,24cyclictest2739-21apt-get11:57:151
631899331,29cyclictest18086-21sed12:12:221
631899331,27cyclictest0-21swapper/111:32:161
631899330,2cyclictest0-21swapper/113:35:141
631899330,2cyclictest0-21swapper/111:22:231
631899330,29cyclictest21488-21unixbench_singl13:22:361
633499321,28cyclictest11346-21tr11:32:183
6318993213,16cyclictest28782-21expr08:57:161
631899320,2cyclictest20784-21apt-get11:42:151
631899320,2cyclictest20784-21apt-get11:42:151
631899320,2cyclictest0-21swapper/113:17:061
631899320,2cyclictest0-21swapper/113:17:061
631899320,2cyclictest0-21swapper/108:52:001
95562310,1sleep00-21swapper/009:22:290
633499313,22cyclictest11994-21apt-get08:22:123
633499310,5cyclictest10090-21taskset09:26:563
631899315,22cyclictest17158-21grep08:32:141
631899311,26cyclictest12341-21cut08:22:151
631899311,26cyclictest1027-21grep12:27:331
6318993112,16cyclictest1760-21runrttasks09:24:341
6318993111,16cyclictest11259-21unixbench_singl08:17:291
631899310,2cyclictest0-21swapper/113:12:341
631899310,2cyclictest0-21swapper/111:37:151
631899310,2cyclictest0-21swapper/111:37:151
631899310,2cyclictest0-21swapper/108:52:121
631899310,28cyclictest2736-21/usr/sbin/munin13:37:281
6313993126,3cyclictest9-21ksoftirqd/011:02:230
633499307,20cyclictest13257-21hddtemp_smartct09:32:183
633499301,5cyclictest31421-21grep09:02:163
631899301,25cyclictest14973-21apt-get11:02:161
631899301,23cyclictest29358-21apt-get11:17:181
6318993011,16cyclictest660-21gdbus08:37:231
631899300,2cyclictest0-21swapper/109:57:211
631899300,2cyclictest0-21swapper/109:57:211
631899300,26cyclictest22042-21taskset09:52:011
6313993025,3cyclictest9-21ksoftirqd/009:47:180
6313993024,3cyclictest9-21ksoftirqd/012:47:210
6313993024,3cyclictest9-21ksoftirqd/010:47:020
6313993024,3cyclictest9-21ksoftirqd/010:02:010
6313993024,3cyclictest9-21ksoftirqd/008:16:580
633499295,20cyclictest14498-21grep08:27:143
633499295,20cyclictest1009-21cut09:07:163
631899291,24cyclictest1302-21gltestperf09:07:191
631899290,25cyclictest14434-21/usr/sbin/munin08:27:141
6313992924,3cyclictest9-21ksoftirqd/013:37:060
6313992922,4cyclictest9-21ksoftirqd/011:27:160
633499281,5cyclictest27566-21grep10:02:153
633499281,5cyclictest27566-21grep10:02:153
633499281,3cyclictest17883-21tune2fs09:42:163
6313992822,3cyclictest9-21ksoftirqd/012:27:040
6313992822,3cyclictest9-21ksoftirqd/012:02:230
6313992822,3cyclictest9-21ksoftirqd/010:12:010
6313992722,3cyclictest9-21ksoftirqd/013:37:200
6313992722,3cyclictest9-21ksoftirqd/013:27:240
6313992722,3cyclictest9-21ksoftirqd/011:47:030
6313992722,3cyclictest9-21ksoftirqd/011:47:030
6313992722,3cyclictest9-21ksoftirqd/010:37:020
6313992722,3cyclictest9-21ksoftirqd/010:17:220
6313992722,3cyclictest9-21ksoftirqd/009:47:000
6313992722,3cyclictest9-21ksoftirqd/009:17:230
6313992722,3cyclictest9-21ksoftirqd/009:07:000
6313992721,3cyclictest9-21ksoftirqd/012:07:200
6313992721,3cyclictest9-21ksoftirqd/009:52:160
6313992720,4cyclictest9-21ksoftirqd/010:06:490
6313992720,4cyclictest9-21ksoftirqd/010:06:490
6313992715,9cyclictest9-21ksoftirqd/013:12:220
6313992621,3cyclictest9-21ksoftirqd/011:32:350
6313992621,3cyclictest9-21ksoftirqd/010:57:030
6313992621,3cyclictest9-21ksoftirqd/009:32:200
6313992621,3cyclictest9-21ksoftirqd/008:47:130
6313992620,3cyclictest9-21ksoftirqd/013:22:350
6313992620,3cyclictest9-21ksoftirqd/012:17:190
6313992620,3cyclictest9-21ksoftirqd/008:54:510
6313992615,6cyclictest9-21ksoftirqd/008:42:110
6313992521,2cyclictest9-21ksoftirqd/013:07:240
6313992520,3cyclictest9-21ksoftirqd/011:52:200
6313992520,3cyclictest9-21ksoftirqd/011:37:230
6313992520,3cyclictest9-21ksoftirqd/011:37:230
6313992520,3cyclictest9-21ksoftirqd/011:12:200
6313992520,3cyclictest9-21ksoftirqd/008:41:590
6313992520,3cyclictest9-21ksoftirqd/008:29:420
6313992519,4cyclictest9-21ksoftirqd/008:26:250
6313992519,4cyclictest9-21ksoftirqd/008:19:100
6313992519,3cyclictest9-21ksoftirqd/009:37:110
6313992518,4cyclictest9-21ksoftirqd/009:13:500
6313992419,3cyclictest9-21ksoftirqd/008:36:590
6313992418,3cyclictest9-21ksoftirqd/012:52:220
6313992418,3cyclictest9-21ksoftirqd/010:00:270
6313992318,3cyclictest9-21ksoftirqd/011:23:050
6313992318,3cyclictest9-21ksoftirqd/009:27:430
631399224,2cyclictest121rcu_preempt12:46:250
6313992217,3cyclictest9-21ksoftirqd/010:47:300
6313992216,3cyclictest9-21ksoftirqd/010:57:080
6313992116,3cyclictest9-21ksoftirqd/012:17:040
6313992110,5cyclictest9-21ksoftirqd/013:17:190
6313992110,5cyclictest9-21ksoftirqd/013:17:190
6313992014,3cyclictest9-21ksoftirqd/009:08:430
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional