Skip to content
This repository was archived by the owner on Jun 20, 2024. It is now read-only.

Log objects via JSON for better debugging #2666

Merged
merged 4 commits into from
Dec 7, 2016
Merged

Log objects via JSON for better debugging #2666

merged 4 commits into from
Dec 7, 2016

Conversation

@bboreham
Copy link
Contributor

@bboreham bboreham commented Nov 24, 2016

Fixes #2633

@bboreham bboreham added this to the 1.9.0 milestone Nov 24, 2016
@awh awh self-assigned this Nov 25, 2016
@awh
Copy link
Contributor

awh commented Nov 25, 2016 •
edited
Loading

LGTM, but it prompts me to suggest some other changes (NB these were issues before, but the increased size of log lines now we're actually logging the full objects brings things to a head):

  • The default logrus 'logfmt' format is injecting millions of backslashes which need to be stripped out if you want to parse the JSON - perhaps we should use the common/logging.go formatter
  • Log lines are truly huge now - I'm still minded to leave them in for a bit longer, but perhaps we should demote them to DEBUG, set the default log level to DEBUG but expose an override

Thoughts? Should these go in separate PR(s)?

@awh awh assigned bboreham and unassigned awh Nov 25, 2016
@bboreham bboreham force-pushed the issues/2633-json-events branch from c5450ee to a799248 Compare December 6, 2016 17:39
@bboreham bboreham force-pushed the issues/2633-json-events branch from a799248 to e7f423c Compare December 6, 2016 17:40
@bboreham
Copy link
Contributor Author

bboreham commented Dec 6, 2016

My, that was a lot of backslashes. I put the Weave formatter in, so now we get:

INFO: 2016/12/06 17:23:47.092667 adding entry 10.38.0.2 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG
INFO: 2016/12/06 17:23:47.461555 EVENT AddPod {"metadata":{"name":"kube-apiserver-host1","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kube-apiserver-host1","uid":"bf13e2f0-bbd8-11e6-87c4-42010a800002","resourceVersion":"424","creationTimestamp":"2016-12-06T17:23:47Z","labels":{"component":"kube-apiserver","tier":"control-plane"},"annotations":{"kubernetes.io/config.hash":"0446d05fb9406214210e8d29397f8bf2","kubernetes.io/config.mirror":"0446d05fb9406214210e8d29397f8bf2","kubernetes.io/config.seen":"2016-12-06T17:22:20.379055536Z","kubernetes.io/config.source":"file"}},"spec":{"volumes":[{"name":"certs","hostPath":{"path":"/etc/ssl/certs"}},{"name":"pki","hostPath":{"path":"/etc/kubernetes"}}],"containers":[{"name":"kube-apiserver","image":"gcr.io/google_containers/kube-apiserver-amd64:v1.4.0","command":["/usr/local/bin/kube-apiserver","--v=4","--insecure-bind-address=127.0.0.1","--etcd-servers=http://127.0.0.1:2379","--admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota","--service-cluster-ip-range=100.64.0.0/12","--service-account-key-file=/etc/kubernetes/pki/apiserver-key.pem","--client-ca-file=/etc/kubernetes/pki/ca.pem","--tls-cert-file=/etc/kubernetes/pki/apiserver.pem","--tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem","--token-auth-file=/etc/kubernetes/pki/tokens.csv","--secure-port=443","--allow-privileged","--etcd-servers=http://127.0.0.1:2379"],"resources":{"requests":{"cpu":"250m"}},"volumeMounts":[{"name":"certs","mountPath":"/etc/ssl/certs"},{"name":"pki","readOnly":true,"mountPath":"/etc/kubernetes/"}],"livenessProbe":{"httpGet":{"path":"/healthz","port":8080,"host":"127.0.0.1","scheme":"HTTP"},"initialDelaySeconds":15,"timeoutSeconds":15,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","nodeName":"host1","hostNetwork":true,"securityContext":{}},"status":{"phase":"Pending"}}

Still pretty long.

On the subject of the default log level, I guess we could add a --log-level flag and default it to debug, then change the default to info at some later date?

@bboreham bboreham force-pushed the issues/2633-json-events branch from e7f423c to 71bf040 Compare December 7, 2016 11:54
@bboreham
Copy link
Contributor Author

bboreham commented Dec 7, 2016

I have converted weave-npc to use common logging, moved pod events to DEBU and added a --log-level flag

@bboreham bboreham assigned awh and unassigned bboreham Dec 7, 2016
@awh awh merged commit 11ff188 into master Dec 7, 2016
@awh awh deleted the issues/2633-json-events branch December 7, 2016 17:25
This was referenced Dec 13, 2016
Closed
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

@awh awh

Labels

None yet

Projects

None yet

Milestone

1.9.0

Development

Successfully merging this pull request may close these issues.

3 participants

@bboreham @awh